Development of the self driving patrol robot

Development of the self driving patrol robot

On the 9th, HL Mando signed a memorandum of understanding (MOU) with AJ Daewon on the commercialization of artificial intelligence (AI) self-driving patrol robots and decided to deploy the self-driving patrol robot ‘Golly’ in Raemiansur Apartment in Gwacheon. Golly has been given a new mission following Villa Village in Gwanak-gu, Seoul! Today, we prepared a time to look back on the journey until the patrol robot Golly was born.

The development of Golly began in 2019 when an in-house contest was held to propose a new business. I was looking for an item to apply for the contest, and I wanted to select a topic that could solve socially insufficient infrastructure through robots. Then, I came across the problem of lack of patrol personnel and the resulting public security vacuum.

The number of people in charge per police officer in our country is 400 as of 2021, which is quite high compared to the average of around 350 in OECD member countries. In 2019, the number of people in charge per police officer was 422.

The installation of CCTVs to fill the public security vacuum is increasing, but there is a limit to reducing the number of crimes, and the number of security guards is also increasing year by year. However, the working environment of security workers is not easily improved. Therefore, we thought that if a robot could replace the simple part of moving and checking in the patrol task of checking and judging special situations while moving in a designated area, it would make up for the lack of manpower, and patrol robot 'Golly' '. So in 2020, we started developing Golly, a full-scale outdoor patrol robot.

In the early days of patrol robot development, the project had to be carried out with a very small number of people. That's why we started to actively outsource. We actively sought out robot-related university laboratories, start-ups, and mid-sized companies across the country, and attended all robot-related exhibitions to collect information on market trends and related companies.

Even if it is difficult to select a partner company, the development period is almost doubled due to a sudden change in the person in charge of the partner company, or the parts of the driving platform purchased overseas for rapid development are damaged and the motor performance is degraded. there was

Therefore, parts with insufficient durability had to be newly manufactured with increased rigidity, and the robot had to be remodeled to make the robot lighter for the performance of the insufficient motor.

When developing Golly II, many improvements were made based on the experience in Golly I. The driving platform was collaborated with a domestic company whose durability was verified, and the controller for the driving motor was developed separately through a motor control specialized company. In this process, we made every effort to select partners such as university laboratories and specialized companies, and had to go through twists and turns, such as development failures and delays in the process of replacing partners.

Despite facing various difficulties, development of the goalie continued. Now, it is time to apply technologies suitable for ‘autonomous driving patrol robots’. Unlike indoors, Golly's main purpose is outdoor driving with many variables, and since it has to drive in environments such as alleys and parks rather than roads, enabling autonomous driving without existing infrastructure such as roads, lanes, and traffic lights was a big challenge.

For outdoor self-driving robots, the accuracy of localization, which allows the robot to determine its own location, is considered an important technology factor. The robot must know exactly where it is and where it is going before it can start driving. One of the most frequently used location recognition technologies in outdoor environments is the method of using the ‘Global Navigation Satellite System (GNSS)’. GNSS is a technology that many people commonly call GPS, and you can easily see that GPS is a part of GNSS.

Before starting mobile robot development, our team was in charge of pre-development of Lv.4 autonomous driving technology. We were developing technology to enable fully autonomous driving of a self-driving vehicle called ‘HOCKEY’ on general roads in Pangyo, and the high-precision GNSS technology used here was applied to Golly.

The next task is to create a map of the driving environment for autonomous driving of mobile robots. Goli applied 'SLAM (Simultaneous Localization And Mapping)' technology. Goli creates maps through 3D lidar, and Lidar SLAM technology allows more information about the outdoor environment to be captured on the map.

On the hardware side, we paid attention to the selection of surveillance cameras for the night patrol function. Since patrol robots are no different from moving CCTVs, camera performance is very important. The Goli is equipped with several cameras, but firstly, a PTZ (Pan-Tilt-Zoom) camera from a CCTV specialist was adopted to cover a wider area. In addition, through infrared photography, the visibility at night reached 200m, and a separate thermal camera was also installed. Thermal cameras are especially effective for night patrols, as they can capture areas that are difficult to monitor with PTZ cameras or 360° cameras.

Self-driving robots encounter various obstacles while driving. In the alleyway, which is the main activity stage of Golly, there are various obstacles that can collide with the robot, such as parked vehicles, moving vehicles, motorcycles, and bicycles along with numerous pedestrians. In crowded parks and apartment complexes, children and pets who are curious about robots may approach them. Therefore, the running speed was set to the speed of a person walking quickly so that the goalie does not receive a large impact even if bumped into, and a soft cover was applied to minimize damage in the event of an unexpected shock.

In Golly II, several problems found through trial operation have been improved. First, for more accurate location recognition, the existing GNSS-only based system was changed to use GNSS and LIDAR together. If you rely only on GPS (GNSS), it is difficult to recognize the location in an environment where GPS is not well caught, but if you use LIDAR, you are a little more free from these environmental restrictions. In addition, the dual 2D/3D lidar sensor applied to the goalie recognizes obstacles within a range of 30m so that the camera mounted on the goalie does not cause problems in autonomous driving even if the lens is covered by foreign substances.

Next, an omni-wheel that can move in all directions was applied to the front wheel of the goalie. The application of the omni wheel enables more free direction change and the small turning radius makes it easy to change direction even in narrow spaces such as alleys. Normally, small-sized casters used for free change of direction can get caught in gaps in subways and drainage covers in outdoor environments, which can become an obstacle to driving. And you can change direction more freely.

The robot design was also changed. Golly I focused on the patrol function and was designed in a structure in which mission equipment cannot be detached. Through this, robots can be used for various purposes depending on the type of mission equipment, and the optimal combination of parts is possible according to the customer's situation.

Now, we need to design the goalie's work process so that the goalie can work smoothly at the patrol site. Golly can operate for up to 8 hours or more on a single charge, and can travel through autonomous driving in a designated section. However, it is still legally impossible for self-driving robots to walk alone, so a person must accompany them. Therefore, we decided to set a time to operate the goalie and have it patrol missions at a specific time.

Take, for example, patrolling an apartment complex. At the set time, the goalie will patrol the designated patrol area within the complex. The videos captured during the patrol process are transmitted to the control system of the management office of the apartment, and the control personnel of the management office check the situation of the patrol site through the video. In certain sections, guidance broadcasts, etc. can be transmitted. When a special situation occurs, broadcasting or mutual communication is possible through the robot directly from the control center. When the patrol is over, the goalie returns to the storage room to recharge.

Real-time communication with the control center is important for Golly's patrol process. This required solving the problem of communicating large amounts of data. Goli was developed to transmit data through 5G communication. Because 5G communication can transmit large amounts of data quickly, images taken by multiple cameras installed in the goalie can be transmitted to the control center in real time. Of course, we couldn't develop communication problems on our own, so we collaborated with domestic carriers. Goli operated by Baegot Life Park solved the data communication problem through collaboration with LG U+ and Goli operated by Gwanak-gu with SKT.

Now it's time to put the golly on patrol and prove it. Along with the development of Goli I, we conducted a pilot operation at Baegot Life Park with the cooperation of Siheung City. However, it was not possible to operate Golly in the park only with the cooperation of Siheung City. According to the Parks and Greenery Act, a mobile robot must weigh less than 30 kg and have a maximum speed of less than 25 km per hour in order to move around the park. Goli was unable to operate because it weighed over 200 kg. These issues were addressed through regulatory sandbox exceptions. Based on the cooperation of the local government and related organizations, Baegot Life Park became the only place where pilot operation was carried out from Goli I to Goli II, and it was an opportunity to improve many problems found in Goli I.

The pilot operation in Gwanak-gu with Goli II started as part of a national project called ‘R&SD Leading Project for People’s Sympathy and Public Participation’. Incheon University became the host organization and carried out the task of ‘development of self-driving single female household safe patrol service technology’, and HL Mando, SKT, and Gwanak-gu Office participated. Incheon University was in charge of autonomous driving software, Mando was in charge of patrol robot HW, and SKT was in charge of video control and communication development. Patrols were carried out in studio-dwelling areas near Seoul National University, and after the project is over, additional operations are being carried out in the area between Byeolbitnaerincheon (Dorimcheon) and Sillim Station and in Nakseongdae Park near Nakseongdae Station.

However, there are still some regrets and limitations to overcome in the goalie.

The biggest disappointment in the functional part of Golly is that Golly still needs people. When charging and starting, human intervention is required, such as connecting the charger directly or turning on the engine. Since the ultimate reason for developing Golly is to reduce the burden of manpower, we plan to improve it so that more tasks can be performed automatically by introducing an automatic charging function in the future.

Golly can also collect special situations, but judgment on this still needs to be done by humans. For example, in the case of smoking, a thermal camera can detect a heat source, but it is still difficult for the robot to determine whether the cause is smoking or not. Therefore, it seems that it will still take a long time for robots to directly judge illegal situations in order to guide life order. Instead, we are developing a way to specify a high-crime area and broadcast guidance on patrol in that area.

The institutional aspect is even more disappointing. According to the Road Traffic Act, outdoor mobile robots are regulated as ordinary vehicles. Therefore, operation on sidewalks and crosswalks is not possible. The area you can drive to patrol is limited to roads only. Also, as explained earlier, it is a pity that a companion is required when driving a goalie. This is because we believe that fully autonomous driving with minimal human intervention in a given patrol environment will ultimately be possible for patrol tasks using patrol robots such as Golly. This part is expected to be resolved soon as there is a recent movement to amend the law recognizing self-driving robots as pedestrians.

In addition, video recording is essential for patrol robots to perform a function called ‘patrol’. However, according to the Personal Information Protection Act, it is prohibited to store or transmit the video recorded by the individual without the consent of the individual. In addition, there are still many regulations that goalies must overcome, such as regulations on the size and weight of mobile robots according to the Parks and Greenery Act described above.

In order to solve various regulations that must be overcome for the commercialization of mobile robots, we are raising our voices by participating in the ‘Autonomous Robot Alliance’. The autonomous robot alliance, which is managed by the Korea Robot Industry Association, is joined by many companies that are developing mobile robot services, including HL Mando, LG Electronics, KT, Hyundai Robotics, and Woowa Brothers. Here, various efforts are being made, such as holding monthly meetings and delivering opinions to the National Assembly and the media to revise the Intelligent Robot Act to address the various regulations described above.