Links

Abstract

The invention discloses a method based on monocular vision for detecting and roughly positioning the edge of a road, and relates to the field of machine vision and intelligent control. Aiming at a continuous road with different edge characteristics, two road edge detection methods are supplied and suitable for semistructured and nonstructured roads and can be applied to vision navigation and intelligent control over a robot. The invention provides a method for detecting the edge of the road based on colors and a method for detecting the edge of the road based on threshold value partitioning. On the basis of the obtained edge of the road, an image is subjected to inverted perspective projection transformation, so that a front view is transformed into a top view; and according to a linear corresponding relation between a pixel and an actual distance in the image, a perpendicular distance from the current position of the robot to the edge of the road and a course angle of the robot can be calculated. The method is easy to implement, high in anti-interference performance, high in instantaneity and suitable for the semistructured and nonstructured roads.

[0002] 智能移动机器人是包含环境感知、控制决策、路径规划等多个模块并应用计算机技术、信息技术、机器人技术、微电子技术的复杂系统。 [0002] intelligent mobile robot is included situational awareness, decision-making control multiple modules, route planning and application of complex computer systems technology, information technology, robotics, microelectronics technology.它一般融合了多种传感器技术，包括GPS、激光雷达、摄像机、超声波、陀螺仪等，将感知的数据进行处理和融合，从而得到机器人定位信息，前方障碍信息，用于行走规划和决策。 It typically combines multiple sensor technologies, including GPS, laser radar, a camera, an ultrasonic, a gyroscope, the sensed data is processed and fused to obtain location information of the robot, the front obstacle information for travel planning and decision making.移动机器人现在广泛应用于军事、医疗、农业、エ业等各个领域。 Mobile robots are now widely used in military, medical, agriculture, Ester industry.特别是在遇到极端环境和危险条件时，如核污染、高压、深海，甚至是外太空，移动机器人均可以有效降低人工作业的风险，提高工作效率，推动了社会和科技的进步与发展。 Especially in the face of extreme environmental and hazardous conditions, such as nuclear pollution, high pressure, deep sea, or even outer space, the mobile robot can be effectively reduce the risk of manual work, improve efficiency, and promote social progress and development and science and technology .

[0003] 移动机器人有多种分类，由控制类型可分为遥控型机器人和自主型机器人。 [0003] Mobile robots have multiple classification can be divided by a remote control type robot and autonomous robots.

[0004] 2011年11月26日，美国成功发射了“好奇号”新一代火星探测车是集最新科技的核动カ遥控型移动机器人，它是继“勇气号”、“机遇号”、“凤凰号”之后第四个火星探测车，具有很强的移动行驶能力，携帯了大量环境分析监测仪器，可以将多种分析资料通过卫星传回地球。 [0004] November 26, 2011, the United States successfully launched the "Curiosity" is the next-generation Mars rover set of nuclear latest technology ka remote control mobile robots, it is the "Spirit", "Opportunity", " after the Phoenix "Mars rover fourth, has a strong ability to move running, carrying a lot of Bands environmental analysis and monitoring instruments can analyze a variety of data via satellite back to Earth.

[0006] 其中智能汽车又是目前备受关注的移动机器人技术应用领域之一。 [0006] where the Smart car is one of the applications of mobile robot technology is closely watched.清华大学研制的智能汽车THMR-V，可以行驶在一般道路和高速公路上，设计车速为高速公路80km/h，一般道路20km/h。 Tsinghua University developed the Smart car THMR-V, can travel on a general road and highway design speed of the highway 80km / h, the general road 20km / h.汽车上装备有彩色摄像机和激光测距仪，差分GPS、磁罗盘和光码盘等导航和环境检测系统，可在校园等非结构化环境中行驶和避障。 Equipped with a color camera and laser range finder on the car, differential GPS, magnetic compass and optical encoder such as navigation and environmental monitoring system, driving and obstacle avoidance in school and other unstructured environment.谷歌于2010年公布了自己正在研制中的智能车，丰田Prius。 Google announced its own smart car being developed in 2010, the Toyota Prius.它安装了视频摄像头，激光测距仪，雷达传感器，GPS卫星定位系统，从传感信息中获得交通情况，在谷歌自己的地图导航下行驶了超过16万英里(其中包括偶尔有人控制的行驶路段)。 It installed a video camera, laser range finder, radar sensors, GPS satellite positioning system, access to traffic information from the sensor, traveling more than 160,000 miles under its own Google Maps Navigation (including occasional control of someone driving route ).智能车感知道路环境是依靠多种传感器的信息融合，对传感器的精度要求较高。 Smart car road environment is perceived to rely on a variety of sensor information fusion, higher sensor accuracy.

[0007] 对于移动机器人来说，最常使用的是视觉传感器，即安装彩色摄像机。 [0007] For a mobile robot, the vision sensor is the most commonly used, i.e. color camera installation.计算机视觉，模式识别技术对移动机器人的导航起到了至关重要的作用。 Computer vision, pattern recognition technology for mobile robot navigation played a crucial role.对于结构化道路的视觉识别算法，一般是对车道线，交通标识，信号灯及道路区域的检测。 For visual recognition algorithm structured way, typically the detection of lane lines, traffic signs, signals and road area.对于非结构化道路的视觉识别算法，有对于可行驶区域，障碍物的检测。 For visual recognition algorithm unstructured road, there travelable area for the obstacle detection.目前带有车道线的结构化的道路检测算法已经比较成熟，而对于乡村道路、不带车道线的城市道路的检测还缺少ー些通用性较强的算法。 Currently road detection algorithm with structured lane line is relatively mature, and for rural roads, urban roads without detecting the lane line is missing ー these highly versatile algorithm.

[0008] 针对目前的多数视觉识别算法，对传感器要求高，时间复杂性高的缺点，对非结构化的环境识别仍需要满足较多条件，无法准确的给出导航信息的现状，本发明提出了基于单目视觉的道路边缘检测及粗定位方法。 [0008] for the most current visual recognition algorithms, high sensor requirements, the time complexity of high disadvantage, unstructured environment recognition still need to meet more conditions can not give the exact status of navigation information, the present invention has rough road edge detection and location based on monocular vision.发明内容 SUMMARY

[0010] 本发明的特征在干，是ー种半自动式的基于移动机器人单目视觉的道路边缘检测和粗定位方法，是在由操作人员操控的计算机和装在移动机器人上的单镜头CCD摄像机共同串接而成的基于单目视觉的道路边缘检测和粗定位系统中实现的，该方法是依次按照以下步骤实现的； [0010] The features of the present invention when dry, is ー kinds of semi-automatic road edge detection and the coarse positioning method for mobile robot based on monocular vision, it is manipulated by the operator computer and mounted on a mobile robot single lens CCD cameras together road edge detection and the coarse positioning system based on concatenation of monocular vision implemented, according to the process is sequentially implemented steps;

[0011] 步骤（I)，单镜头CCD摄像机将机器人前进时拍摄到的能反映道路和非道路边界两侧颜色差异的路况图像输入到计算机中显示； [0011] Step (I), a single lens CCD camera captured the robot proceeds to reflect the difference in color on both sides of the road and non-road road boundary image is input to the computer display;

[0012] 步骤（2)，操作人员根据所述的道路和非道路边界两侧的颜色差异，来选择性的启动设在所述计算机内的两个不同的软件模块； [0012] Step (2), in accordance with the operator and road color difference on both sides of the non-road boundary, arranged to selectively start the computer in two different software modules;

[0013] 对于道路和非道路边界两侧颜色有显著差异的RGB顔色空间彩色路况图像，启动基于颜色的道路边缘检测模块； [0013] There are significant differences for both road and off-road color boundary road color image the RGB color space, based on the color start road edge detection module;

[0014] 对于道路和非道路边界两侧颜色模糊且无明显边界的RBG顔色空间彩色路况图像，启动基于阈值分割的道路边缘检测模块； [0014] For both sides of the road and non-road boundary color blur and no obvious boundaries traffic RBG color space color image, segmentation of the start threshold based on the road edge detection module;

·[0015] 步骤（3)，所述基于顔色的道路边缘检测模块依次执行以下步骤进行道路边缘的检测； * [0015] Step (3), based on the color of the road edge detection module performs the steps of sequentially detecting the edge of the road;

[0017] 步骤（3. 2)，从所述彩色路况图像中取出感兴趣区域ROI的图像并转换到HSV顔色空间，得到新的路况图像； [0017] Step (3.2), taken from the region of interest ROI in the image and the color image traffic HSV color space conversion to obtain new road image;

[0023] 设定步骤（3. 2)得到的结果图像边缘连接处作为控制边缘其梯度阈值为50，内侧初始分割处作为控制强边缘其梯度阈值为150，输入为所述感兴趣区域ROI的HSV颜色空间图像和上述两个參数；输出为canny抽边图； [0023] Step set (3.2) The results obtained at the connection edges of the image as the control edge 50 of its gradient threshold value, the initial segmentation at the inner edge thereof as the control of a strong gradient threshold value of 150, the inputs of the region of interest ROI HSV color space image and the two parameters; FIG outputs canny edge extraction;

[0026] 步骤（3. 6),利用openCV库中的cvDilate函数模块对步骤（3. 5)得到的结果图像进行膨胀操作，输出为经过腐蚀膨胀操作后的緑色或土黄色区域结果图像； [0026] Step (3.6), using the results of the image library cvDilate openCV step function module (3.5) was subjected to dilation, or khaki green output image area subjected to corrosion result of the expansion operation;

[0027] 步骤（3. 7)，从步骤（3. 6)得到的结果图像中提取拟合用的边缘点：为道路左侧边缘时，在步骤（3. 6)得到的结果图像内，从右至左地扫描，若所述边缘区域内某个边缘点左侧的5个像素内有绿色点,而右侧的5个像素内对应的没有绿色点,则认为该边缘点为真实边缘点，所述边缘点左右不足5个像素的像素点直接舍去，得到ー个真实边缘点样本集；对土黄色区域按同样步骤； [0028] 步骤（3. 8)，判断真实边缘点： [0027] Step (3.7), the image extraction result obtained from step (3.6) is fitted with edge points: the left edge of the road when the resulting image at step (3.6) is obtained, from right to left scanning, if the green pixel within 5 points of a point in the left edge of the edge region, and the corresponding pixels to the right of the five points is not green, the edge is considered the true edge point points, the edge points is less than 5 pixels around pixels directly discarded, to obtain a true edge point ー sample set; region of khaki by the same step; [0028] step (3.8), determining the true edge point:

[0029] 若所述真实边缘点样本集中，真实边缘点的个数m > 30，则执行以下步骤； [0029] If the true edge point in the sample set, the number m of the true edge point> 30, perform the following steps;

[0031] 步骤（3. 8. 2),通过openCV库中的cvFitline函数模块,用最小ニ乘法拟合出一条直线；在计算误差类型为CV_DIST_L2，极坐标半径和角度误差均为O. 01条件下，输出所拟合直线的參数；之后再统计与所述直线距离小于4个像素点的边缘点个数k，以此作为拟合用的点； [0031] Step (3. 8.2), by cvFitline openCV library function modules, with a minimum fitting a straight line multiplication ni; type of error in calculating CV_DIST_L2, radius and polar angle error conditions are O. 01 , the output parameters of the straight line fit; then, after the statistical distance is less than four straight edge point number k of pixels, as a point of fitting;

[0032] 步骤（3. 8. 3)，若k/m大于O. 3，则接受拟合模型，再通过所述的k个点利用最小ニ乘法重新拟合直线，即得到最終結果；否则，返回步骤（3.8. I);当重复执行次数大于设定的最大循环次数200次时，则失败；若所述真实边缘点样本集中的真实边缘点数小于30时则失败； [0032] Step (3. 8.3), if k / m is greater than O. 3, fitting model is accepted and re-fitting a straight line, i.e., the final result obtained by using the minimum multiplying said ni k points; otherwise returns to the step (3.8 I.); when the maximum number of iterations is repeatedly performed 200 times greater than the set number of times, the fail; true edge point if the sample set is less than the true edge point 30 fails;

[0033] 步骤（4)，所述基于阈值分割的道路边缘检测模块，在道路右侧边缘处，依次做以下步骤进行道路边缘检测； [0033] Step (4), based on the road edge detection threshold segmentation module, at the edge of the road on the right, the following steps may be sequentially road edge detection;

[0034] 步骤（4. I)，设置感兴趣区域ROI的大小为图像右下角1/4部分； [0034] Step (4. I), the region of interest ROI is set to the lower right quarter of the size;

[0035] 步骤（4. 2)，取出所述感兴趣区域图像，并按下式转换为灰度图像； [0035] Step (4.2), removing the image region of interest, and the following formula into a gray image;

[0037] X，y代表所述图像中以左上角为原点的像素点坐标值；V(I表示（X，y)像素点的灰度值，R表示彩色图像中所在像素的红色通道值，G表示彩色图像中所在像素的緑色通道值，B表示彩色图像中所在像素的蓝色通道值，v0, R，G，B都在[0，255]内； [0037] X, the pixel coordinates of the upper left corner as the origin of the value y represents the image; V (I represented by (X-, y) is the gradation value of the pixel, R represents the red channel value of the color image where the pixels, G represents green channel image where the pixel color value, B represents a color image where the pixels in the blue channel values, v0, R, G, B in [0,255] therein;

[0038] 步骤（4. 3)，按下式对像素点（x，y)进行均值模糊处理，将以（x，y)为中心的5X5的像素值矩阵V与核矩阵K卷积，得到新的像素值Vl ； [0038] Step (4.3), the following equation for the pixel (x, y) mean-blurring process, will be (x, y) is the pixel value of the center matrix V 5X5 convolution with the kernel matrix K to give new pixel value Vl;

[0042] 步骤（4. 4)，计算每个灰度值对应的像素点数，制成顔色直方图； [0042] Step (4.4), calculates the number of pixels corresponding to each gray value, color histogram is made;

[0043] 步骤（4. 5)，设定CCD摄像机获取的所述感兴趣区域图像，在灰度空间中的值域为 [0043] Step (4.5), setting the CCD camera acquiring images of the region of interest, in a range of gradation space

[0044] 按下式计算评判值η (t)，遍历所有灰度值的评判值，选出评判值最大的一个灰度值作为分类最优的灰度阈值； [0044] The evaluation value is calculated as η (t), traversing all the gradation values ​​of the evaluation values, selecting a maximum evaluation value as a gradation value of the gradation optimal classification threshold;

[0048] ρ,为灰度值i的像素点出现的概率；图像的总像素为 [0048] ρ, the probability of the pixel gray values ​​i occurring; total pixels of the image

，n表示灰度值为i的像素点数，利用直方图的结果计算得到灰度为i的像素出现的概率为Pi = f; , N-i represents the number of pixels of the gray value, calculated from the results obtained using a histogram of the gradation occurrence probability of pixel i Pi = f;

[0049] P0W^P1 (t)是根据所述灰度阈值t划分的两类的灰度值出现概率；μ T(t)为整个灰度值值域内像素点的概率加权总和； [0049] P0W ^ P1 (t) is the value of the occurrence probability of the divided two t gradation according to the gradation threshold value; weighted summation μ T (t) is the probability of pixel values ​​within the entire gray value;

[0050] [0050]

[0051] ol(t)为根据灰度阈值t划分的大于和小于等于该值的两类像素点，通过计算出现概率 [0051] ol (t) is divided according to the gray threshold value t is greater than two and less than or equal to the pixel value, by calculating the probability

[0052] 其中，0|(t)为根据灰度阈值t划分的大于和小于等于该值的两类像素点，通过计算出现概率PtlUhP1⑴而得到的类间方差；而得到的类内方差； [0052] wherein, 0 | (t) is divided according to the gray threshold value t is greater than two and less than or equal to the pixel value variance between two classes by calculating the occurrence probability obtained PtlUhP1⑴; within-class variance is obtained;

[0058] 步骤（4. 6),利用所述openCV库中的cvCanny函数模块对步骤（4. 5)得到的ニ值图像进行canny抽边，输出为canny抽边图像；设定作为边缘连接处的控制边缘连接的像素梯度阈值为50，作为边缘初始切割处的控制强边缘连接处的像素梯度阈值为100 ； Ni binary image [0058] Step (4.6), using a library cvCanny openCV step function module (4.5) was subjected to extraction canny edge, an output image is a canny edge extraction; set as an edge connection gradient threshold pixel value of a control edge connector 50, the pixel intensity gradient threshold as a control at the cutting edges of the initial value of the connection 100;

[0059] 步骤（4. 7)，对于步骤（4.6)得到的ニ值化处理以后的canny抽边图，按照从左到右取得第一个白色边缘点，作为边界点，得到一个边界点集； [0059] Step (4.7), for step (4.6) ni values ​​obtained after processing of FIG canny edge extraction to obtain a white first edge point from left to right, as a boundary point, boundary point to obtain a set of ;

[0060] 步骤（4. 8),利用openCV库中的CvHoughLines函数模块对步骤（4. 8)得到的边界点集中的所有边界点进行Hough直线拟合，得到道路边缘；设定极坐标半径的分辨率为3个像素点，极坐标角度分辨率为$，直线转换到极坐标上需通过它的曲线条数最少为50 ； [0060] Step (4.8), using a library CvHoughLines openCV step function module (4.8) for all boundary points of the set of boundary points obtained Hough straight line fitting performed to obtain road edges; the set polar angle resolution of 3 pixels, a resolution of polar angle $, the linear conversion to polar coordinates by the number of its required minimum of curve pieces 50;

[0061] 步骤（5)，按ー下步骤进行机器人和道路边缘的相对粗定位； [0061] Step (5), the next step according ー relatively coarse positioning robot and the edge of the road;

[0062] 步骤（5. I)，把步骤（4)检测得到的路况边缘的前向图通过做逆透视投影变换得到道路边缘的俯视图，步骤如下： [0062] Step (5. I), the step (4) the front edge of the road to FIG detection obtained by making inverse perspective projection conversion to obtain a plan view of the road edge, the following steps:

[0063] 步骤（5. I. I)，用机器人上的摄像机拍摄一幅带有梯形图像，在俯视图中为矩形的前向图； [0063] Step (5. I. I), taken with the camera on the robot with an image trapezoidal, rectangular in plan view in FIG forward;

[0065] 步骤（5. I. 3)，任意设置俯视图中矩形大小与像素的对应关系，从而得到所述四个顶点在做完逆透视投影变换后得到的俯视图中的四个顶点，记录坐标； [0065] Step (5. I. 3), arbitrarily set a correspondence relationship with the pixel size of the rectangle in plan view, resulting in a plan view of the four vertices of four vertices in the finished inverse perspective projection conversion is obtained, the coordinates of the recording ;

[0069] 所述航向角偏差是指机器人行驶方向AC与道路边缘线的延长线AE之间的夹角Θ , [0069] The deviation of the heading angle refers to the angle between the traveling direction of the robot and a road edge line of the AC extension line AE Θ,

[0070] 所述横向位移是指所述机器人行驶方向AC上的机器人位置B到所述道路边缘延长线上的垂直距离BD; [0070] The lateral displacement of the robot refers to a position B on the traveling direction of the robot to the AC extended vertical distance from the road edge line BD;

[0072] 在所述前向图的道路边缘线上任意两点，将其中的每ー个点的坐标乘以所述逆透视投影变换矩阵H，就得到所述道路边缘线的延长线上对应两点的坐标（Xl，yi)、(x2, J2)，相应的直线AE 的斜率k= G2I2V(X2-X1),贝丨J Θ =90° -BrctanGO^X1=X2JlJ θ=0。 [0072] any two points on the edge line of the road map in the front, in which the coordinates of each point ー multiplying said inverse perspective projection transformation matrix H, to obtain an extension line of the edge line corresponding to the road two coordinates (Xl, yi), (x2, J2), the slope of the line corresponding to the AE k = G2I2V (X2-X1), Tony Shu J Θ = 90 ° -BrctanGO ^ X1 = X2JlJ θ = 0.； ;

[0074] 步骤（5. 2. 2. 1)，设定：所述俯视图的左上角为坐标原点，坐标用像素点表示，每ー个像素点对应实际长度为丨mm，在步骤（5. I. 3)中设置俯视图坐标时设定的；联立如下方程可得到机器人行驶方向AC中A的纵坐标为h ； [0074] Step (5. 2. 2. 1), setting: a top left corner of the graph is the origin of coordinates, the ordinate represents a pixel, each pixel point corresponds ー actual length of Shu mm, in step (5. I. 3) set in plan view in FIG coordinate set; ordinate simultaneous equations can be obtained as the traveling direction of the robot a is AC in H;

[0075] [0075]

表示图像宽度， It denotes image width,

[0076] 步骤（5. 2. 2. 2)，设机器人行驶方向AC与所述俯视图的交点C的纵坐标为图像高度height,则从C点到道路边缘线的延长线的垂直距离为CE=Sin( Θ )* oc *(h_height)； [0076] Step (5. 2. 2. 2), the robot is provided with AC and ordinate direction intersecting point C is a plan view of the height image height, the vertical distance from the point C to the extension line of the edge line of the road CE = Sin (Θ) * oc * (h_height);

[0079] d是机器人中心B和俯视图下边缘的垂直距离BC，也是机器人前方的盲区距离，单位为mm。 [0079] d is a plan view of a robot and a center B of the lower edge of the vertical distance of the BC, but also in front of the robot blind distance, in mm.

[0080] 本发明的主要优点是： [0080] The main advantage of the present invention are:

[0081] I、本发明针对连续的具有不同边缘特征的道路提供了两种道路边缘检测方法，对道路环境的适应性强，通用性好。 [0081] I, the present invention provides two way continuous road edge detection method for edge having different characteristics, adaptability to the road environment, common good.ー种适用于道路和非道路边界两侧颜色差异比较大的情况(如两侧为绿化带的道路)，对光照条件和阴影的敏感性较弱；另ー种方法适用于边界处相对不够清晰的道路情况，对阴影很敏感。ー species suitable for color differences on both sides of the boundary road and off-road is relatively large (e.g., a road on both sides of the green band), sensitivity to weak lighting conditions and shadows; ー Another method suitable for the boundary is not clear enough at the opposite the road conditions are very sensitive to the shadows.两种方法主要适用于半结构化道路以及非结构化道路。 Both methods mainly applied to semi-structured and unstructured way road.

[0082] 2、有些时候由于摄像机的视野或者安装角度，或者道路较宽等原因，图像不能完整显示道路的两侧，本发明的道路边缘检测方法针对单侧的边缘检测，降低了对摄像机视野范围和安装角度的要求；本发明的方法不需要分辨率很高的摄像机，降低了对摄像机清晰度的要求。 [0082] 2, since the field of view of the camera in some cases, or installation angle or wide road and other reasons, the image can not be displayed completely on both sides of the road, a road edge detection method of the present invention is directed to a one-sided edge detection, reduces the field of vision of the camera and mounting angle requirements; method of the present invention does not require high resolution cameras, the camera reduces the requirement for clarity.

[0083] 3、本发明在检测到道路的边缘后，对机器人进行了相对于道路边缘的粗定位，给出了定位結果，此结果可以更好的辅助控制机器人的行迸。 [0083] 3, the present invention after an edge is detected in the road, the robot relative to the edge of the road rough positioning, the positioning result is given, this can result in better control of the robot Beng adjuvant.

[0084] 本发明实现了两种基于单目视觉的道路边缘检测方法，在其基础上，实现了机器人相对于道路边缘的粗定位。 [0084] The present invention implements two road edge detection method based on monocular vision, on its basis, to achieve a coarse positioning of the robot relative to the edge of the road.

[0085] 第一种方法是基于顔色的道路边缘检测方法，首先将图像转换到HSV顔色空间上，利用道路和非道路的顔色差异，以H值为主范围区间，S和V为辅助范围区间，将图像粗略分成道路和非道路区域的ニ值灰度图；结合原始图像的canny边缘图，根据道路边缘处梯度较大以及边缘两侧的颜色不同，提取道路边缘点；当道路边缘点的数目较多吋，才进行下一歩的处理，否则丢弃这ー帧图像对下ー帧图像进行处理；然后用RANSAC方法对道路边缘点进行直线拟合。 [0085] A first method is a road edge detection based on color, the image is first converted into the HSV color space, using the color difference of road and off-road to the main window interval value H, S and V is an auxiliary range section , the image is roughly divided into the grayscale values ​​of ni and road-road area; FIG conjunction canny edge of the original image, the larger the gradient and a different color on both sides of the edge, the edge point extraction according to the road edge of the road; a road when the edge points larger number of inches before the next processing ho, or discard the frame image processed ー ー the frame image; road edge points and fitting a straight line with the RANSAC method.从已有的道路边缘点中随机抽取两点进行直线拟合，当拟合结果较好时，去除干扰较大的点，对剩余的边缘点再次进行随机抽样直线拟合。 Randomly selected from two o'clock in the existing road edge points fit a straight line, fitting is better when a result, the interference removing larger dots, the remaining edge points in a random sample of a straight line fit again.两次拟合可以使拟合结果更准确，并且能很好的去除图像中一些噪点的干扰。 Two fitting can be made more accurate fit, and can well remove some of the noise in the noisy images.

[0086] 另ー种方法是基于阈值分割的道路边缘检测方法，先将原图像转化成灰度图像；选择灰度阈值t将图像划为道路和非道路两类，计算两类的类间方差；遍历t的值，选取使得类间方差最大的t值作为道路区域和非道路区域的临界值；利用灰度阈值t将图像ニ值化，提取ニ值化图像的边缘点即为道路边缘点；对道路边缘点进行Hough直线检测，得到道路边缘线。 [0086] Another method is to ー road edge detection threshold segmentation method based on first original image is converted into grayscale image; selecting a threshold gray image t road and off-road classified as two types, class variance between the two calculated ; traversal value t, such that the selection of the maximum between-class variance value t and the threshold value as the road-road area region; t by using the image gray threshold value of ni, ni extracted edge points is the binarized image of the road edge points ; road edge points Hough line detection, to obtain the road edge line.

[0087] 在检测到道路边缘之后，对机器人进行和道路边缘的相对粗定位，计算机器人和道路方向的夹角以及机器人与道路边缘的垂直距离。 [0087] After detecting the edge of the road, the robot and the relative coarse positioning edge of the road, calculating an angle of the robot and the direction of the road and the vertical distance of the robot and the edge of the road.对已得到的道路边缘线图像进行逆透视投影变化，将图像从摄像机正常的视角图转换为俯视图；逆透视投影变换后的图像和实际的距离有ー个线性的对应关系，一个像素点在实际中代表的长度为α毫米；利用图像中直线的斜率可以算出机器人当前和道路方向的夹角Θ，机器人和道路边缘的垂直距离δ可以从图像中根据三角关系求出。 Performing inverse perspective projection changes road edge line image has been obtained, the images from the camera normal perspective view of a transition is a plan view; image and the actual distance after inverse perspective projection conversion has ー linear correspondence between a pixel in an actual Representative of length α mm; slope of the straight line with the image of the robot can be calculated current direction of the road and the angle Θ, the robot and the vertical distance δ can be determined from the edge of the road image on the basis triangle.

[0088] 本发明的优劣：本发明提出的道路边缘检测方法适用于半结构化和非结构化道路，对于图像稳定性及传感器色彩偏差上有很好的鲁棒性，可用于小型移动机器人。 [0088] The merits of the present invention: a road edge detection method proposed by the present invention is applied to semi-structured and unstructured road, and for the stability of the image sensor with a color deviation is robust, can be used in small mobile robot .同吋，可以很好满足实时导航的需求，处理延迟低，对于非结构化道路适用范围广。 With inch, can well meet the needs of real-time navigation, low processing delay, applicable for a wide range of unstructured road.算法得出的最終結果，可以辅助机器人定位，对机器人控制决策起到决定性作用。 The final results of the algorithm derived, may be assisted robot positioning, the robot control decisions play a decisive role.由于算法基于单目视觉，因此具有视觉算法的通病，即对于光照条件、阴影较为敏感，在这些条件下的检测效果会受到影响。 Since the algorithm based on monocular vision, vision algorithms therefore have a common problem, i.e., the conditions for lighting, shadows more sensitive detection effect under these conditions will be affected.

附图说明 BRIEF DESCRIPTION

[0089] 图1、2为基于顔色的道路边缘检测算法的步骤说明图，其中：[0090] 图I (a)为划分ROI后的原始图像，图I (b)为canny抽边结果图，图I (C)为绿色区域提取结果图，图1(d)为腐蚀膨胀处理后的緑色区域图。 [0089] FIG. 2 is an explanatory diagram based on the color of the road edge detection algorithm step, wherein: [0090] FIG I (a) dividing the original image the ROI, FIG I (b) is a canny edge extraction result FIG. FIG I (C) of FIG green region extraction result, FIG. 1 (d) of FIG processed green region of erosion and dilation.

[0093] 图3(a)为划分ROI后的原始图像，图3(b)为转化为灰度空间的图像。 [0093] FIG. 3 (a) dividing an original image to the ROI, FIG. 3 (b) is an image space into gray.

[0094] 图4(a)为利用灰度阈值进行ニ值化的结果图，图4(b)为canny抽边后去冗简化的结果图。 [0094] FIG. 4 (a) is performed using Ni gray threshold value of the result, and FIG. 4 (b) is a canny edge extraction to the redundant results simplified FIG.

[0095] 图6、7、8、9为机器人粗定位实施方法的步骤说明图，其中： [0096] 图5为最终直线拟合结果图。 [0095] FIG 6,7,8,9 explaining a step of the method of the robot coarse positioning embodiment, wherein: [0096] FIG. 5 is the final straight line fitting results of FIG.

[0097] 图6为世界坐标系与图像坐标系示意图。 [0097] FIG. 6 is a world coordinate system and the image coordinate system. FIG.

[0098] 图7为对于实际道路，逆透视投影变换前后的效果对比图。 [0098] FIG. 7 is an actual road, before and after comparison of the effect of inverse perspective projection transformation.其中（a)为原图，（b)为逆透视投影变换后的图。 Wherein (a) is the original image, (b) after the inverse perspective projection conversion FIG.

[0100] 图9为航向角偏差和横向位移计算不意图。 [0100] FIG. 9 is a calculated heading angle deviation and is not intended to lateral displacement.

[0101] 图10为基于单目视觉的道路边缘检测和粗定位方法的流程图。 [0101] FIG. 10 is a flowchart of edge detection based on monocular vision roads and rough positioning method.

具体实施方式 detailed description

[0102] 本发明设计了基于单目视觉的道路边缘检测和粗定位方法。 [0102] The present invention contemplates rough road edge detection and location based on monocular vision.图I所示的是道路边缘检测和粗定位的流程。 Road edge detection and rough location of the flow I shown in FIG.本发明设计了两种道路边缘检测方法，一种是基于颜色的道路边缘检测方法，另ー种是基于阈值分割的道路边缘检测方法。 The present invention is designed in two road edge detection method for a road edge detection based on color, the other species is ー road edge detection method based on threshold segmentation.在道路边缘检测的基础上，对机器人进行了相对于道路边缘的粗定位。 On the basis of the road edge detection, the robot is positioned relative to the edge of the road coarse.实现过程中的图像处理利用的是openCV开源库。 Image processing implementation process is openCV use of open source libraries.

[0104] 首先根据实际道路环境选择用哪ー种算法，如果道路和非道路边界两侧颜色差异比较大的情况(如两侧为绿化带的道路）就选择基于颜色的道路边缘检测方法；否则就选取基于阈值分割的道路边缘检测方法。 [0104] First, according to the actual road conditions ー selection algorithms which, if the road and off-road color differences on both sides of the boundary is relatively large (e.g., a road on both sides of the green band) to select road edge detection method based on color; otherwise, to select the road edge detection method based on threshold segmentation.

[0105] ニ、道路边缘检测算法 [0105] Ni, road edge detection algorithm

[0106] 基于颜色的道路边缘检测方法 [0106] Based on the color of the road edge detection method

[0107] 本方法的关键是道路边缘点的提取和道路边缘线的拟合。 Key [0107] The present method is the extraction fitting and a road edge line of a road edge points.由于道路边缘两侧的颜色一般是有很大区别的，并且道路边缘处的图像梯度较大，因此本方法使用颜色范围区间和canny图像相结合的方法提取边缘点；提取的边缘点可能有一些干扰，本方法使用抗干扰カ较强的RANSAC方法来拟合道路边缘线。 Since the road edge on both sides of the color is generally very different, and the image gradient at the edge of the road is large, so the present method interval and the range of colors using the method of combining images canny edge point is extracted; extracted edge points may be some interference, the method using a strong interference ka RANSAC method to fit a road edge line.以两旁是绿化带的道路为例，具体的步骤如下： Road lined with green belt, for example, specific steps are as follows:

[0119] 对于RGB顔色空间的彩色图像，对于每个像素点进行上述计算，可以转化为HSV颜色空间图像。 [0119] For a color image the RGB color space, the above calculation for each pixel, can be converted into the HSV color space image.

[0120] 3.进行canny抽边得到canny图：利用openCV库中的cvCanny函数进行，输入參数为原始图像，作为控制边缘的梯度阈值为50，内侧初始分割处作为控制强边缘的像素阈值为150，输出为canny抽边图；图I (a)为划分ROI后的图像，图1(b)为取出ROI后的canny抽边图； [0120] 3. canny edge extraction obtained canny FIG: cvCanny using openCV library function, the input parameters for the original image, a gradient threshold value of the control edge 50, at the inner side of the initial segmentation as a strong edge pixel control threshold value 150, the output is drawn canny edge; Figure I (a) dividing the image after the ROI, FIG. 1 (b) pumping canny edge view of the ROI after withdrawal;

[0122] 5.对绿色或土黄色区域图进行腐蚀操作：利用openCV库中的函数cvErode函数，输入为原始图像，其余參数为默认的矩形矩阵，无需输入，输出为结果图像； [0122] The area of ​​green or ocher FIG etching operation: using a function cvErode openCV library function, the input original image, the other parameters as default rectangular matrix, without the input and output of the image result;

[0123] 6.对绿色或土黄色区域图进行膨胀操作：利用openCV库中的函数cvDilate函数，输入为原始图像，其余參数为默认，无需输入，输出为结果图像；如图1(d)； [0123] 6. FIG green or yellow soil region inflated operation: using a function cvDilate openCV library function, the input original image, the other parameters as default, no input, output result of the image; FIG. 1 (d) ;

[0124] 7.提取用于拟合的边缘点：对canny图从下至上从右至左扫描（如果是道路左侧边缘)，若canny图中的边缘点左侧5像素内有緑色点或土黄色点，右侧5像素内没有緑色点或者土黄色点,则认定为真实的边缘点。 [0124] 7. A fitting for extracting edge points: canny FIG oriented to scan from right to left (if the left edge of the road) from the bottom, 5 points in the green pixel to the left edge point if the drawing or canny khaki points, within 5 pixels to the right spot or not green khaki point is identified as the true edge point.一行最多只有ー个真实边缘点。 One line at most ー a real edge point.左右不足5像素的点直接舍去； Point less than about 5 pixels directly discarded;

[0125] 8.如果真实边缘点的个数小于30则无法拟合；如果真实边缘点的个数大于30，则采用RANSAC算法对上面步骤得到的真实边缘点进行直线拟合： [0125] 8. If the number of true edge point is less than 30 can not be fitted; true edge point if the number is greater than 30, then the true edge point using the RANSAC algorithm obtained in the above step of fitting a straight line:

[0126] a)设真实边缘点样本集中总共有m个点，随机地选出两个样本数据； [0126] a) the true edge point in the sample set is provided with a total of m points, two randomly selected sample data;

[0127] b)通过这两个点用最小ニ乘法拟合直线：利用openCV库中的CvFitLine函数,输入为两个点的点集，计算误差类型为CV_DIST_L2，极坐标半径和角度误差均为0.01，输出为直线的參数；然后统计与直线距离小于4像素的点即拟合上的点的个数K ； [0127] b) multiplying the minimum Ni fitting a straight line through these two points: CvFitLine using openCV library function, two input point set points, calculation error type CV_DIST_L2, polar radius and angle errors are 0.01 , linear output parameter; point count is then less than the linear distance of 4 pixels, i.e., the number of fitting points K;

[0128] c)如果K/m大于0. 3就接受拟合模型，则接受该拟合结果，并且通过所有能够拟合上的数据利用最小ニ乘方法拟合直线；否则重新执行a)_c)； [0128] c) If K / m is larger than 0.3 accept fitting model, the fitting result is accepted, and the minimum Ni by using the method of fitting a straight line through the data can be fitted on all; otherwise re-execute a) _c );

[0129] d)如果重复执行次数大于最大允许次数200，则失败； [0129] d) If the number is greater than the maximum allowed is repeatedly performed 200 times, the failure;

[0130] 最终结果如图2。 [0130] The final result is shown.

[0131] 基于阈值分割的道路边缘检测算法[0132] 针对ー些非结构化道路中，道路区域边缘不清晰，道路区域与非道路区域顔色差异大的特点，在系统中利用了ー种最大类间方差法，将摄像机捕捉到的图像，在灰度空间中进行自适应的聚类，使其动态地进行阈值分割，从而识别出可行驶的道路区域。 [0131] Road thresholding edge detection algorithm based on [0132] For some ー unstructured road, a road edge of the region is not clear, the non-road area with a large difference in color characteristics of a road area, using a maximum ー species in the system class variance between the captured camera image, in gray adaptive spatial clustering, segmentation so that the threshold value is dynamically, thereby identifying a road travelable area.

[0133] 算法中将道路区域和非道路区域分为两类Ctl和C1，通过分别计算两类的类间方差和类内方差，获得最佳灰度分割阈值t，即灰度阈值t可使得两类的类间方差最大，类内方差最小。 [0133] Algorithm for road and off-road in the region divided into two regions and Ctl C1, by calculating the variance between the two classes and class variance to obtain optimal gamma segmentation threshold value t, i.e., the gradation may be such that the threshold value t min-max variance within-class variance between the two types of classes.该方法的判决条件，是根据将图像二分吋，同一类的像素点顔色相似，不同类的像素点顔色差异大的特点演化而来。 Decision conditions of the process, based on a half-inch image, similar to the same kind of color pixels, the color pixels large difference in the characteristics of different types evolved.

[0134] 具体实施步骤如下： [0134] In particular embodiments the steps of:

[0135] I.设置感兴趣区域ROI大小为图像右下角四分之一部分。 [0135] I. disposed region of interest ROI size as the bottom right quarter of the image.图3(a)为划分感兴趣区域后得到的图像； FIG. 3 (a) dividing the image region of interest is obtained;

[0136] 2.将图像转化为灰度图像，公式如下(其中X，y代表图像中以左上角为原点的像素点坐标值，Vtl表示U，y)像素点的灰度值，R表示彩色图像中所在像素的红色通道值，G表示彩色图像中所在像素的緑色通道值，B表示彩色图像中所在像素的蓝色通道值，V0, R，G，B e [0,255])： [0136] 2. The image into a gray image, the following formula (wherein X, y, the representative image pixel coordinate value of the upper left corner as the origin, represents the U-Vt1, y) is the gradation value of the pixel, R represents the color pixels in the image where the red channel value, G represents a color image where the pixel value of the green channel, B represents a pixel where the color image in the blue channel value, V0, R, G, B e [0,255]):

[0137] [0137]

[0138] 3.利用均值模糊算法对图像进行模糊处理，用以减少个别的顔色突变点。 [0138] 3. Using the mean fuzzy algorithm for image blurring process for reducing the color of individual point mutations.图3(b)为灰度空间的图像；具体公式如下(其中X，y代表图像中以左上角为原点的像素点坐标值，V。表示（x，y)像素点的灰度值，K表示均值模糊的核矩阵，V表示以（x，y)为中心的5X5的像素值矩阵，*表示卷积）： FIG 3 (b) is the image gray space; in particular the following formula (wherein X, y, the representative image pixel coordinate value of the upper left corner as the origin, V represents (x, y) of the pixel gray scale value, K. It represents the mean blur kernel matrix, V represented by (x, y) as the center of the 5X5 matrix of pixel values, * represents convolution):

[0143] 5.摄像机获取的图像在灰度空间中的值域为[0，1-1] (I e [0，255])，设灰度值为i的像素点个数为IV则图像的总像素为N = Eljnp则利用直方图的结果可计算得到灰度为i的像素出现的概率为 [0143] The range of the camera image acquired in the gray space is [0,1-1] (I e [0,255]), the number of pixels arranged gradation value IV as the image i of total pixels N = Eljnp the calculated results obtained using a histogram gradation occurrence probability of pixel i

[0144] [0144]

[0145] 所以根据上述公式我们可以得到： [0145] Therefore, the above equation we get:

[0146] [0146]

[0147] 对于某ー个灰度值t作为阈值，可将灰度值划分为两类。 [0147] For a gray value t ー as a threshold value, the gradation value may be divided into two categories.通过以下公式可计算得 Calculated by the following equation obtained

到两类的出现概率为： The probability of two types:

[0149] 因此，可以计算得到两类的均值，以及图像的总体灰度概率均值分别为： [0149] Thus, the mean can be calculated in two, and the overall probability that the gradation image mean values:

[0151] 由此可得出两类的类间方差为： [0151] thus be drawn between two categories of variance:

[0158] 6.按照上一歩得到灰度分类阈值，将高于此数值和低于此数值的两类像素进行简单的黑白ニ值处理。 [0158] 6. The gradation obtained in the previous classification threshold ho, this value will be higher than this value and below two types of black and white pixels ni simple handling.结果如图4(a)。 Results As shown in FIG 4 (a).公式如下(其中x，y代表图像中以左上角为原点的像素点坐标值，V1表示（x，y)像素点ニ值化之前的灰度值，V表示所在像素ニ值化处理后的灰度值，t为上ー步求出的灰度分类阈值）： The following formula (wherein the pixel values ​​represent the image point coordinates x, y to the upper left corner as the origin, V1 represents (x, y) is the gradation value of the previous pixel value of ni, V represents a pixel where the gray value processing ni values, t is the classification threshold ー gradation steps obtained):

则咖）=1:¾¾ The coffee) = 1: ¾¾

[0160] 7.对此ニ值图像进行canny抽边：利用openCV库中的cvCanny函数进行,输入參数为原始图像，作为控制边缘的梯度阈值为50，内侧初始分割处作为控制强边缘的像素阈值为150，输出为canny抽边图 [0160] 7. The value of this image ni canny edge extraction: openCV performed using cvCanny library function, the input parameters for the original image, a gradient threshold value of the control edge 50, at the inner side of the initial segmentation of a strong edge pixel control the threshold value of 150, the output of FIG canny edge extraction

[0161] 8.根据上一歩的canny抽边图，按行从左到右取得第一个白色点（如果是沿道路右边缘)，得到一个边界点集，起到去除冗余信息简化结果的作用。 [0161] The suction on a ho the canny edge graph, the first row from left to right to obtain a white point (along the road if the right edge), to give a set of boundary points, simplified functions to remove redundant information results effect.图4(b)为抽边且边缘简化后的结果； FIG. 4 (b) and the result of edge extraction of the edge simplified;

[0162] 9.利用上ー步得到的边界点集进行Hough直线拟合：使用openCV库中的cvHoughLines函数，输入为原始图像，极坐标半径的分辨率3像素，极坐标系角度的分辨率 [0162] 9. A set of points on the boundary using the Hough ー step was subjected to linear fitting: Use cvHoughLines openCV library function, the input original image, the resolution of the polar coordinate radius 3 pixels, a resolution angle polar coordinate system

t，直线所需最少的曲线交点50，输出为Hough拟合图。 Minimum required number t, the intersection of the straight line curve 50, the output fitting Hough FIG.即得到了道路边沿。 That is the way to get the edge.如图5。 5.

[0163] 三、机器人和道路边缘的相对粗定位 [0163] Third, the robot and the opposite edge of the road coarse positioning

[0164] 机器人行进时，需不断的通过摄像机检测出的道路边缘来计算当前机器人和道路边缘之间的横向位移以及航向角偏差。 [0164] When the robot travels, we need to be constantly detected by the camera to calculate the current road edge lateral displacement between the robot and the road edges and the heading angle deviation.横向位移即为机器人中心与道路边缘的垂直距离。 It is the lateral displacement of the robot from the vertical center of the edge of the road.航向角偏差即为机器人当前行驶方向和欲行驶的规划路线之间的夹角。 Heading angle deviation is the angle between the traveling direction and the current directions of the robot to be traveling.具体的计算步骤如下：[0165] I、逆透视投影变换 The specific calculation steps are as follows: [0165] I, inverse perspective projection transformation

[0166]为了方便的计算出机器人的横向位移偏差和航向角偏差，将检测到的车道边缘图像做逆透视投影变换，将前向图变换成俯视图，如图7。 [0166] In order to facilitate the lateral displacement deviation calculated angular deviation and heading of the robot, the detected lane edge image do inverse perspective projection transformation, converted into a forward plan view of FIG, 7 as shown in FIG.

[0167] 在欧氏空间中定义两个坐标系W和I，分别表示世界坐标系和图像坐标系： [0167] defines two coordinate systems I and W in the Euclidean space, the world coordinate system respectively, and the image coordinate system:

[0168] W= {(x, y, z)} e E3 [0168] W = {(x, y, z)} e E3

[0169] I= {(U，V)} e E2 [0169] I = {(U, V)} e E2

[0170] 其中E是欧氏空间，χ、Y、z是世界坐标系W中的坐标，Z=O的平面是地平面；u、V是图像坐标系中的坐标，图像坐标系的原点是图像的左上角。 [0170] where E is the Euclidean space, χ, Y, z are the coordinates of the world coordinate system W is, Z = O plane is a ground plane; u, V are the coordinates of the image coordinate system, the origin of the image coordinate system is the upper left corner of the image.

[0171] 逆透视投影变换的实质就是将图像坐标系I下的道路图像，即摄像机所获取的图像，变换到世界坐标系W下的Z=O平面中，两者关系如图6。 [0171] inverse perspective projection conversion is the essence of the road image in the image coordinate system I, i.e., images acquired by a camera is converted to the Z = O plane of the world coordinate system W, the relationship between the two in Figure 6.

[0172] 由于Z=0，并且变换后的图像没有旋转变换，因此只需要将ー个3X3的变换矩阵和原图像相乘就可以得到逆透视投影变换后的图像。 [0172] Since Z = 0, and not the transformed image rotational transformation, and therefore only needs to ー a 3X3 transformation matrix and the original image can be obtained by multiplying the inverse perspective projection image transformation.

[0173] f(x, y) = (x, y, I) XH [0173] f (x, y) = (x, y, I) XH

[0174] 其中（X，y)是原始图像中一点的坐标，H是3X3的逆透视投影变换矩阵，f (X，y)是变换后的坐标。 [0174] where (X, y) is the coordinates of a point in the original image, H is the inverse of the perspective projection transformation matrix is ​​3X3, f (X, y) is transformed coordinates.

[0175] 具体步骤如下： [0175] the following steps:

[0176] (I)用机器人上的摄像机拍摄一幅带有梯形图案，俯视图时为矩形的前向图； [0176] (I) captured by a camera on the robot with a trapezoidal pattern, a rectangular top plan view of FIG forward;

[0177] (2)在图片中取ー个梯形的四个顶点，记录坐标； [0177] (2) Take four vertices ー a trapezoid in the picture, recording the coordinates;

[0178] (3)根据俯视图中的矩形大小与像素的对应关系设置矩形这四个顶点在新的坐标系中的χ，y坐标； [0178] (3) arranged in a rectangular four vertices χ new coordinate system based on the correspondence and the pixel size of the rectangle in plan view, y coordinates;

[0179] (4)利用openCV中的cvWarpPerspectiveQMatrix函数求出逆透视投影矩阵H。 [0179] (4) an inverse perspective projection matrix is ​​determined using the cvWarpPerspectiveQMatrix openCV function H.cvffarpPerspectiveQMatrix函数的输入是上述两套坐标构成的两个4X2的矩阵,输出是逆透视投影变换矩阵H。 CvffarpPerspectiveQMatrix input function is a matrix composed of the above-described two sets of coordinates 4X2, the output is the inverse perspective projection transformation matrix H.

[0181 ] 用于计算逆透视投影变换矩阵的图像和逆透视投影变化后的图像如图8. [0181] is used to calculate the inverse perspective projection image and the perspective projection transformation matrix of the inverse variation of the image shown in FIG 8.

4个点的横纵坐标组成的，就是变换前和变换后的坐标。 Horizontal and vertical coordinates of the four points of the composition, is the coordinate before conversion and after conversion.求得的逆透视投影变换矩阵为： -0.291892 -1.36039 487.334 Obtained by the inverse perspective projection transformation matrix is: -1.36039 -0.291892 487.334

[0185] 由于图片中一个方砖的实际宽度为600mm，各顶点在图像中的坐标也已知，可算出逆透视投影变换后一个像素点在实际中的距离是7. 5_，这是后续计算机器人行驶状态的重要依据。 [0185] As the actual width of the image in a tiling is 600mm, the coordinates of each vertex in the image are also known, can be calculated after inverse perspective projection conversion from a pixel in practice is 7. 5_, which is subsequent calculation robot important basis for running state.由于逆透视投影变换后的图像的四个点的坐标是人为规定的，因此可以控制这四个点的位置来调整图像中一个像素点对应的实际距离。 Since the four coordinates of the image points of the inverse perspective projection conversion is artificial and thus the position of the four points can be controlled to adjust the actual distance image corresponding to a pixel.、[0186] 2、航向角偏差和横向位移计算 , [0186] 2, the heading angle error and lateral displacement calculation

[0187] 在图9中，假设机器人的位置是AC中点B的位置，中心实线(直线ABC)是机器人行驶的方向，车道边缘的方向是经过逆透视投影变换后的车道边缘线，虚线是车道边缘线的延长线，它和中心实线(直线ABC)的夹角即为机器人的航向角偏差Θ，线段BD长度就是横向位移。 [0187] In FIG. 9, the position of the assumption that the robot is a position of the AC point B, the center of the solid line (a straight line ABC) is the direction of travel of the robot, the edge of the lane is the lane edge line after the inverse perspective projection transformation, a broken line lane edge line is the extended line, the angle between it and the center solid line (a straight line ABC) of the robot is the heading angle deviation [Theta], the length of the line segment BD is the lateral displacement.

[0188] (I)求行驶角度偏差Θ。 [0188] (I) with angular deviation request Θ.由前面的检测算法可以得到原始图像中道路边缘线上的每一点的坐标，将其乘以逆透视投影变换矩阵H就可以得到直线DE上任意一点的坐标，然后用两点的坐标（X1, Yl)，（X2，y2) (Xi不等于X2)算出直线的斜率k=(y2-y2)/(X2-X1),然后可以求出9=90。 From the foregoing detection algorithm may obtain the coordinates of each point in the original image edge line path, multiplied by the inverse perspective projection transformation matrix H can be obtained at any point on the straight line DE coordinates, then two coordinates (X1, Yl), (X2, y2) (Xi is not equal to X2) calculated from the slope of the straight line k = (y2-y2) / (X2-X1), and 9 = 90 can be determined.-arctan (k);若X1=X2,则0=0。 -arctan (k); if X1 = X2, then 0 = 0.。 .

[0190] i.图中A点的坐标用直线ABC和直线DE的方程联立可以求出来。 [0190] I coordinate. FIG point A can find out with a simultaneous linear equation ABC and the straight line DE.假设图像中左上角为坐标原点，坐标用像素来表示，A点的纵坐标为h。 The image is assumed as the coordinate origin in the upper left corner, the coordinates of the pixel represented by the ordinate of point A to h.方程如下： Equation is as follows:

[0191] [0191]

[0192] ii.在做逆透视投影变换的时候，有一个可选的尺度丨（mm)，即为ー个像素点对应的实际的长度(例如上面提到的7. 5mm)，那么可以得出，线段CE的实际长度为sin( Θ )* oc *(h-height),其中height为图像的像素高度,也就是C的纵坐标。 [0192] ii. In doing inverse perspective projection conversion, when an optional scale Shu (mm), is the actual length ー a corresponding pixel (e.g. 7. 5mm referred to above), it may be obtained the actual length of the line CE is sin (Θ) * oc * (h-height), wherein the height of the image height in pixels, i.e. the ordinate C.

[0193] iii.假设机器人中心和图像下方的实际距离（BC)是d (mm)，这个距离即为摄像机安装在机器人上后的盲区距离，转换成像素的距离就是d/oc，那么横向位移（BD)的计算就变成sin( Θ )* oc * (h-height-d/ )。 [0193] iii. The actual distance and the assumption that the robot below the image center (BC) is d (mm), the distance that is in the blind spot camera mounted on the robot from the rear, the pixel is converted into a distance d / oc, then the lateral displacement (BD) calculation becomes sin (Θ) * oc * (h-height-d /).

Claims (1)

Translated from Chinese

1.基于单目视觉的道路边缘检测和粗定位方法，其特征在于，是一种半自动式的基于移动机器人单目视觉的道路边缘检测和粗定位方法，是在由操作人员操控的计算机和装在移动机器人上的单镜头CCD摄像机共同串接而成的基于单目视觉的道路边缘检测和粗定位系统中实现的，该方法是依次按照以下步骤实现的； 步骤（I)，单镜头CXD摄像机将机器人前进时拍摄到的能反映道路和非道路边界两侧颜色差异的路况图像输入到计算机中显示； 步骤（2)，操作人员根据所述的道路和非道路边界两侧的颜色差异，来选择性的启动设在所述计算机内的两个不同的软件模块； 对于道路和非道路边界两侧颜色有显著差异的RGB颜色空间彩色路况图像，启动基于颜色的道路边缘检测模块； 对于道路和非道路边界两侧颜色模糊且无明显边界的RBG颜色空间彩色路况图像，启动 1. Based on edge detection and the coarse positioning method road monocular vision, wherein a semi-automatic edge detection and rough road location based on monocular vision mobile robot is manipulated by the operator in the computer and installed in single-lens CCD camera on the mobile robot of the concatenation common road edge detection and the coarse positioning system in monocular vision-based implementation, the process is implemented sequentially following steps; step (the I), the single-lens camera CXD captured color differences reflect road and non-road side of the border when the robot forward road image input to the computer display; step (2), the operator by the color difference on both sides of the road and non-road boundary, selected start of two different software modules provided within said computer; significant difference for both road and off-road color boundary road color image the RGB color space, based on the color start road edge detection module; for roads and non- the road on both sides of the border and no color blurring the boundaries of the road RBG color space color image, start基于阈值分割的道路边缘检测模块； 步骤（3)，所述基于颜色的道路边缘检测模块依次执行以下步骤进行道路边缘的检测； 步骤（3. I)，参数初始化，读入所述RBG颜色空间的彩色路况图像，设置感兴趣区域ROI的范围为所述路况图像的1/4〜1/2大小，从所述彩色路况图像左下角开始划分； 步骤（3. 2)，从所述彩色路况图像中取出感兴趣区域ROI的图像并转换到HSV颜色空间，得到新的路况图像； 亮度V=max (R, G, B), fV~min(R,G,B) 丰。 Step (3), based on the color of the road edge detection module performs the following steps sequentially detects a road edge;; based on the road edge detection module thresholding step (3. I), parameter initialization, reads the RBG color space the road color image, the region of interest ROI is set in the range of the road image 1 / 4~1 / 2 size, the color assigned from the bottom left corner of the road image; step (3.2), from the road color remove the region of interest ROI image and converts the image of the HSV color space to give new road image; luminance V = max (R, G, B), fV ~ min (R, G, B) abundance.饱和度S = ^ ， I 0 其他r 60(G - B)/(V - min(R, G, B) if V = R色度H= 120 + |V:=b)丨 詩气240+—^^— ifV= B 、 [V—Ttim(RjGjB)] 若H〈0，则用（H+360)代替，其中RGB分别为原像素点红、蓝、绿三色的亮度值，令Yr =255V，S' =255S，H' =H/2 ; 步骤（3. 3),按以下方式用openCV库中的cvCanny函数模块进行canny抽边，得到canny 图： 设定步骤（3. 2)得到的结果图像边缘连接处作为控制边缘其梯度阈值为50，内侧初始分割处作为控制强边缘其梯度阈值为150，输入为所述感兴趣区域ROI的HSV颜色空间图像和上述两个参数；输出为canny抽边图； 步骤（3. 4)，设定HSV的颜色范围区间，把绿色区域或土黄色区域提取出来，其中绿色的范围区间为（20，0，0)〜(70，40，0)，土黄色的范围为（10，0，0)〜(30，0，150)； 步骤（3. 5),利用openCV库中的cvErode函数模块对步骤（3. 4)得到的绿色或土黄色区域进行腐蚀操作，输 Saturation S = ^, I 0 the other r 60 (G - B) / (V - min (R, G, B) if V = R chroma H = 120 + | V: = b) Shu poetry gas 240 + - ^^ - ifV = B, [V-Ttim (RjGjB)] If H <0, then with (H + 360) in place, wherein the original RGB pixels are red, blue, and green luminance value, so Yr = 255V, S '= 255S, H' = H / 2; in step (3.3), a canny edge extraction with openCV cvCanny function block library in the following manner, to give canny FIG: setting step (3.2) to give the result of the image edge is connected as a control edge which the gradient threshold is 50, the initial segmentation at the inner edge of which a gradient strength as the control threshold 150, an HSV color space input image region of interest ROI and said two parameters; output FIG canny edge extraction; step (3.4), the HSV color range setting section, the extracted area or khaki green region, wherein the green range interval (20,0,0) - (70,40,0 ), khaki range (10,0,0) - (30,0,150); step (3.5), (3.4) obtained using openCV function block library cvErode step green or earth yellow etching operation area, the output为结果图像； 步骤（3. 6),利用openCV库中的cvDilate函数模块对步骤（3. 5)得到的结果图像进行膨胀操作，输出为经过腐蚀膨胀操作后的绿色或土黄色区域结果图像；步骤（3. 7)，从步骤（3. 6)得到的结果图像中提取拟合用的边缘点：为道路左侧边缘时，在步骤（3. 6)得到的结果图像内，从右至左地扫描，若所述边缘区域内某个边缘点左侧的5个像素内有绿色点,而右侧的5个像素内对应的没有绿色点,则认为该边缘点为真实边缘点，所述边缘点左右不足5个像素的像素点直接舍去，得到一个真实边缘点样本集；对土黄色区域按同样步骤； 步骤（3. 8)，判断真实边缘点： 若所述真实边缘点样本集中，真实边缘点的个数m > 30，则执行以下步骤； 步骤（3. 8. I)，从所述真实边缘点样本集中，随机地选出两个真实边缘点，作为两个样本数据； 步骤（3. 8. 2),通过openCV Step (3.6), using the results of the image library cvDilate openCV step function module (3.5) was subjected to dilation, or khaki green output image area subjected to corrosion result of the expansion operation; the result image; step (3.7), the edge points are extracted by fitting the results of the image resulting from step (3.6) of: when the left edge of the road, the result of the image obtained in step (3.6), right to scanning left, if there is an edge within the pixels of a region within 5 points of the left edge of the green dot, and the corresponding pixels to the right of the five points is not green, it is considered that the true edge point is an edge point, the less than five points around said edge pixel point directly discarded, to obtain the true edge point of a sample set; in the same region of yellow soil; step (3.8), determining the true edge point: if the true edge point samples concentration, the number m of the true edge point> 30, perform the following steps; step (3. 8. I), from the true edge point in the sample set, randomly selected two points true edge, as two sample data ; step (3. 8.2), by openCV中的cvFitline函数模块,用最小二乘法拟合出一条直线；在计算误差类型为CV_DIST_L2，极坐标半径和角度误差均为0. 01条件下，输出所拟合直线的参数；之后再统计与所述直线距离小于4个像素点的边缘点个数k，以此作为拟合用的点； 步骤（3. 8. 3)，若k/m大于0. 3，则接受拟合模型，再通过所述的k个点利用最小二乘法重新拟合直线，即得到最终结果；否则，返回步骤（3.8. I);当重复执行次数大于设定的最大循环次数200次时，则失败；若所述真实边缘点样本集中的真实边缘点数小于30时则失败； 步骤（4)，所述基于阈值分割的道路边缘检测模块，在道路右侧边缘处，依次做以下步骤进行道路边缘检测； 步骤（4. I)，设置感兴趣区域ROI的大小为图像右下角1/4部分； 步骤（4. 2)，取出所述感兴趣区域图像，并按下式转换为灰度图像； V0(X，y) =0. 212671XR(x, y) +0. 71516 The cvFitline function module, fitting a straight line by least square method; calculation error in the type of CV_DIST_L2, polar radius and angle errors are conditions under 0.01, the output of the linear fit parameters; Statistics and again after the said linear distance is smaller than four edge point number k of pixels, as a point of fitting; step (3. 8.3), if k / m is greater than 0.3, the fitted model is accepted, then by the k points using the least squares fitting a straight line again, i.e., the final result; otherwise, returns to step (3.8 I.); when the maximum number of iterations is repeatedly performed 200 times greater than the set number of times, then a failure; if the said true edge points in the sample set is less than the true edge point 30 is failed; step (4), based on the road edge detection threshold segmentation module, at the edge of the road on the right, the following steps may be sequentially road edge detection; step ( 4. I), the region of interest ROI set a size of the lower right quarter portion; step (4.2), removing the image region of interest, and the following equation is converted to grayscale; V0 (X, y) = 0. 212671XR (x, y) +0. 715160XG(x, y) +0. 072169XB(x, y)，x，y代表所述图像中以左上角为原点的像素点坐标值；V(1表示（x，y)像素点的灰度值，R表示彩色图像中所在像素的红色通道值，G表示彩色图像中所在像素的绿色通道值，B表示彩色图像中所在像素的蓝色通道值，v0, R，G，B都在[0，255]内； 步骤（4. 3)，按下式对像素点（x，y)进行均值模糊处理，将以（x，y)为中心的5X5的像素值矩阵V与核矩阵K卷积，得到新的像素值Vl ； V1 (X, y) =V*K, * 表示卷积， . +0 072169XB (x, y), x, y represent the coordinates of the image pixel values ​​in the upper left corner as the origin 0XG (x, y); gradation values ​​V (1 represented by (x, y) of the pixel , R represents the red channel value of the color image where the pixels, G represents a green channel image where the pixel color value, B represents the blue color channel values ​​of pixels located in the image, V0, R, G, B are [0, 255] within; step (4.3), the following equation for the pixel (x, y) mean-blurring process, will be (x, y) is the pixel value of the center matrix V 5X5 convolution with the kernel matrix K, and get new pixel value Vl; V1 (X, y) = V * K, * denotes convolution,

按下式计算评判值n (t)，遍历所有灰度值的评判值，选出评判值最大的一个灰度值作为分类最优的灰度阈值； Calculated as the criterion value n (t), traversing all the gradation values ​​of the evaluation values, selecting a maximum evaluation value as a gradation value of the gradation optimal classification threshold;

其中，c)i(t)为根据灰度阈值t划分的大于和小于等于该值的两类像素点，通过计算出现概率匕⑴氺⑴而得到的类间方差； Wherein, c) i (t) is divided according to the gray threshold value t is greater than two and less than or equal to the pixel value, the probability of inter-dagger ⑴ ⑴ Shui-class variance is obtained by calculating the occurrence;

Pi为灰度值i的像素点出现的概率；图像的总像素为 Pi is the probability of the pixel gray values ​​i occurring; total pixels of the image

表示灰度值为i的像素点数，利用直方图的结果计算得到灰度为i的像素出现的概率为 I represents the number of pixels of the tone value, calculated from the results obtained using a histogram probability pixel gradation i is occurring

P0 是根据所述灰度阈值t划分的两类的灰度值出现概率；(t)为整个灰度值值域内像素点的概率加权总和； P0 is the probability of gray value t according to the two divided gray-level threshold; (t) is a weighted sum of the probability values ​​of pixels within the entire gray value;

0?v(t):为根据灰度阈值t划分的大于和小于等于该值的两类像素点，通过计算出现概率而得到的类内方差； 其中，u0(t), yi(t)分别为在阈值t以下和以上的灰度值值域中，像素点出现的概率的加权均值； ? 0 v (t): t is within two pixels of greater than and less than or equal to the divided value, by calculating the probability threshold value according to the gray-class variance is obtained; wherein, u0 (t), yi (t), respectively It is the weighted mean t probability threshold gray value below and above the range, the pixels appears;

步骤（4. 5)，基于步骤（4. 4)得到的灰度分类阈值t，分别把高于所述灰度分类阈值t以及低于所述灰度分类阈值t的两类像素点，将其灰度值按下式进行二值化处理； Step (4.5), based on step (4.4) to give a gray classification threshold t, respectively, the classification threshold is higher than the gray scale t and two gray pixels below the threshold value t classification, the the gray value by the following equation binarizing process;

v(x, y)为进行二值化处理后的像素灰度值； 步骤（4. 6),利用所述openCV库中的cvCanny函数模块对步骤（4. 5)得到的二值图像进行canny抽边，输出为canny抽边图像；设定作为边缘连接处的控制边缘连接的像素梯度阈值为50,作为边缘初始切割处的控制强边缘连接处的像素梯度阈值为100 ； 步骤（4. 7)，对于步骤（4. 6)得到的二值化处理以后的canny抽边图，按照从左到右取得第一个白色边缘点，作为边界点，得到一个边界点集； 步骤（4. 8),利用openCV库中的cvHoughLines函数模块对步骤（4. 8)得到的边界点集中的所有边界点进行Hough直线拟合，得到道路边缘；设定极坐标半径的分辨率为3个像素点，极坐标角度分辨率为$，直线转换到极坐标上需通过它的曲线条数最少为50 ； 步骤（5)，按一下步骤进行机器人和道路边缘的相对粗定位；步骤（5. 1)，把步骤（4)检测得到的路况 v (x, y) is binarized pixel gray value after processing; step (4.6), with the openCV library function module cvCanny binary image obtained in step (4.5) of canny pumping side, the output is canny edge extraction image; gradient threshold is 50 pixels is set as the control edges of the connection at the connector, as the control threshold pixel gradients at the edge of the initial cut at the junction 100 is strong edges; step (4.7 ), for step (4. 6) canny edge evacuated FIG later obtained by binarization process, first obtain a white edge point from left to right, as a boundary point, to obtain a set of boundary points; step (4.8 ), all the boundary points of the boundary point set using openCV library cvHoughLines step function module (4.8) was subjected to Hough fitting a straight line to give the edge of the road; polar angle setting resolution is three pixels, $ polar angle resolution, the linear conversion to polar coordinates by its required minimum number of curvilinear strip 50; step (5), press the coarse positioning step is performed relative to the robot and the edge of the road; step (5.1), the step (4) detecting the resulting traffic边缘的前向图通过做逆透视投影变换得到道路边缘的俯视图，步骤如下： 步骤（5. I. I)，用机器人上的摄像机拍摄一幅带有梯形图像，在俯视图中为矩形的前向图； 步骤（5. I. 2)，在步骤（5.1.1)得到的前向图中选择一个梯形的四个顶点，记录坐标；步骤（5. I. 3)，任意设置俯视图中矩形大小与像素的对应关系，从而得到所述四个顶点在做完逆透视投影变换后得到的俯视图中的四个顶点，记录坐标； 步骤（5. I. 4),利用openCV中的cvWrapPerspectiveQMatrix函数模块，求出逆透视投影变换矩阵H，输入为步骤（5. 1.2)、（5.1.3)中的两套坐标构成的两个4X2的矩阵，输出为逆透视投影变换矩阵H ； 步骤（5. I. 5)，把所述前向图和逆透视投影变换矩阵输入openCV中的cvffrapPerspective函数模块中，得到逆透视投影变换后的俯视图； 步骤（5. 2)，按以下步骤航向角偏差和横向位移： FIG front edge by doing the inverse perspective projection conversion to obtain a plan view of the road edge, the following steps: Step (5. I. I), with a trapezoidal captured forward image, which is rectangular in plan view by a camera on the robot FIG; step (5. I. 2), to select the four vertices of a trapezoid obtained prior to the drawing step (5.1.1), recording the coordinates; step (5. I. 3), arbitrarily set the size of a rectangular plan view correspondence between the pixels, thereby obtaining four vertices of the four vertices of a plan view in the finished inverse perspective projection conversion is obtained, the coordinates of the recording; step (5. I. 4), using the cvWrapPerspectiveQMatrix openCV function module, obtaining an inverse perspective projection transformation matrix H, the input of the step (5 1.2), two matrix coordinates (5.1.3) two 4X2 configuration, the output of the inverse perspective projection transformation matrix H; step (5. the I . 5), the forward map and the inverse perspective projection transformation matrix of the input openCV cvffrapPerspective function module, to give a top view of the inverse perspective projection conversion; step (5.2), the following steps heading angle error and lateral displacement : 所述航向角偏差是指机器人行驶方向AC与道路边缘线的延长线AE之间的夹角e， 所述横向位移是指所述机器人行驶方向AC上的机器人位置B到所述道路边缘延长线上的垂直距离BD ； 步骤（5.2. I)，求航向角偏差9， 在所述前向图的道路边缘线上任意两点，将其中的每一个点的坐标乘以所述逆透视投影变换矩阵H，就得到所述道路边缘线的延长线上对应两点的坐标（Xl，yi)、(x2, y2)，相应的直线AE 的斜率k= G2I2V(X2-X1),贝丨J 0=90。 The heading angle refers to the angular deviation e between the robot and the traveling direction of a road edge line of the AC extension line AE, the lateral displacement of the robot refers to a position B on the traveling direction of the robot AC to the road edge extension line BD on the vertical distance; step (. 5.2 I), the heading angle deviation seeking 9, to the road edge line between any two points of the FIG., each of which is multiplied by the coordinates of points of the inverse perspective projection transformation in the front matrix H, to obtain an extension line of the road edge line corresponding to the two points of coordinates (Xl, yi), (x2, y2), the slope of the straight line corresponding to the AE k = G2I2V (X2-X1), Tony Shu J 0 = 90.-BrctanGO^X1=X2JlJ 0=0。 -BrctanGO ^ X1 = X2JlJ 0 = 0.； 步骤(5. 2. 2)，求横向位移BD, 步骤（5. 2. 2. I)，设定：所述俯视图的左上角为坐标原点，坐标用像素点表示，每一个像素点对应实际长度为~ _，在步骤（5. I. 3)中设置俯视图坐标时设定的；联立如下方程可得到机器人行驶方向AC中A的纵坐标为h ；|A Xl)，width表示图像宽度， 步骤（5. 2. 2. 2)，设机器人行驶方向AC与所述俯视图的交点C的纵坐标为图像高度height,则从C点到道路边缘线的延长线的垂直距离为CE=sin( 0 )* ~ *(h_height)； 步骤（5. 2. 2. 3)，按下式计算横向位移BD ： ; Step (5. 2. 2), lateral displacement of the BD request, step (5. 2. 2. I), is set: a plan view of the upper left corner is the origin of coordinates, the ordinate represents a pixel, each pixel corresponding to the actual length of ~ _, provided plan view coordinate is set in step (5. I. 3); ordinate simultaneous equations can be obtained as the traveling direction of the robot a is AC, h; | a Xl), width represents the image width, step (5. 2. 2. 2), the robot is provided with AC and ordinate direction intersecting point C is a plan view of the height image height, the vertical distance from the point C to the extension line of the road edge line is CE = sin (0) * ~ * (h_height); step (5. 2. 2. 3), the lateral displacement is calculated as BD:

d是机器人中心B和俯视图下边缘的垂直距离BC，也是机器人前方的盲区距离，单位为mm。 d is a plan view of a robot and a center B of the lower edge of the vertical distance of the BC, but also in front of the robot blind distance, in mm.

CN 2012101440822012-05-102012-05-10Method based on monocular vision for detecting and roughly positioning edge of road
CN102682292B
(en)