TechTalks from event: Technical session talks from ICRA 2012

Conference registration code to access these videos can be accessed by visiting this link: PaperPlaza. Step-by-step to access these videos are here: step-by-step process .
Why some of the videos are missing? If you had provided your consent form for your video to be published and still it is missing, please contact support@techtalks.tv

Control and Planning for UAVs

  • Deploying the Max-Sum Algorithm for Decentralised Coordination and Task Allocation of Unmanned Aerial Vehicles for Live Aerial Imagery Collection Authors: Delle Fave, Francesco Maria; Rogers, Alex; Xu, Zhe; Sukkarieh, Salah; Jennings, Nick
    We introduce a new technique for coordinating teams of unmanned aerial vehicles (UAVs) when deployed to collect live aerial imagery of the scene of a disaster. We define this problem as one of task assignment where the UAVs dynamically coordinate over tasks representing the imagery collection requests. To measure the quality of the assignment of one or more UAVs to a task, we propose a novel utility function which encompasses several constraints, such as the task's importance and the UAVs' battery capacity so as to maximise performance. We then solve the resulting optimisation problem using a fully asynchronous and decentralised implementation of the max-sum algorithm, a well known message passing algorithm previously used only in simulated domains. Finally, we evaluate our approach both in simulation and on real hardware. First, we empirically evaluate our utility and show that it yields a better trade off between the quantity and quality of completed tasks than similar utilities that do not take all the constraints into account. Second, we deploy it on two hexacopters and assess its practical viability in the real world.
  • Mixed-Integer Quadratic Program Trajectory Generation for Heterogeneous Quadrotor Teams Authors: Mellinger, Daniel; Kushleyev, Aleksandr; Kumar, Vijay
    We present an algorithm for the generation of optimal trajectories for teams of heterogeneous quadrotors in three-dimensional environments with obstacles. We formulate the problem using mixed-integer quadratic programs (MIQPs) where the integer constraints are used to enforce collision avoidance. The method allows for different sizes, capabilities, and varying dynamic effects between different quadrotors. Experimental results illustrate the method applied to teams of up to four quadrotors ranging from 65 to 962 grams and 21 to 67 cm in width following trajectories in three-dimensional environments with obstacles with accelerations approaching 1g.
  • Safety Verification of Reactive Controllers for UAV Flight in Cluttered Environments Using Barrier Certificates Authors: Barry, Andrew J.; Majumdar, Anirudha; Tedrake, Russ
    Unmanned aerial vehicles (UAVs) have a so-far untapped potential to operate at high speeds through cluttered environments. Many of these systems are limited by their ad-hoc reactive controllers using simple visual cues like optical flow. Here we consider the problem of formally verifying an output-feedback controller for an aircraft operating in an unknown environment. Using recent advances in sums-of-squares programming that allow for efficient computation of barrier functions, we search for global certificates of safety for the closed-loop system in a given environment. In contrast to previous work, we use rational functions to globally approximate non-smooth dynamics and use multiple barrier functions to guard against more than one obstacle. We expect that these formal verification techniques will allow for the comparison, and ultimately optimization, of reactive controllers for robustness to varying initial conditions and environments.
  • On-board Velocity Estimation and Closed-loop Control of a Quadrotor UAV based on Optical Flow Authors: Grabe, Volker; Buelthoff, Heinrich H.; Robuffo Giordano, Paolo
    Robot vision became a field of increasing importance in micro aerial vehicle robotics with the availability of small and light hardware. While most approaches rely on external ground stations because of the need of high computational power, we will present a full autonomous setup using only on-board hardware. Our work is based on the continuous homography constraint to recover ego-motion from optical flow. Thus we are able to provide an efficient fall back routine for any kind of UAV (Unmanned Aerial Vehicles) since we rely solely on a monocular camera and on on-board computation. In particular, we devised two variants of the classical continuous 4-point algorithm and provided an extensive experimental evaluation against a known ground truth. The results show that our approach is able to recover the ego-motion of a flying UAV in realistic conditions and by only relying on the limited on-board computational power. Furthermore, we exploited the velocity estimation for closing the loop and controlling the motion of the UAV online.
  • Visual Terrain Classification by Flying Robots Authors: Khan, Yasir Niaz; Masselli, Andreas; Zell, Andreas
    In this paper we investigate the effectiveness of SURF features for visual terrain classification for outdoor flying robots. A quadrocopter fitted with a single camera is flown over different terrains to take images of the ground below. Each image is divided into a grid and SURF features are calculated at grid intersections. A classifier is then used to learn to differentiate between different terrain types. Classification results of the SURF descriptor are compared with results from other texture descriptors like Local Binary Patterns and Local Ternary Patterns. Six different terrain types are considered in this approcah. Random forests are used for classification on each descriptor. It is shown that SURF features perform better than other descriptors at higher resolutions.
  • Real-Time Decentralized Search with Inter-Agent Collision Avoidance Authors: Gan, Seng Keat; Fitch, Robert; Sukkarieh, Salah
    This paper addresses the problem of coordinating a team of mobile autonomous sensor agents performing a cooperative mission while explicitly avoiding inter-agent collisions in a team negotiation process. Many multi-agent cooperative approaches disregard the potential hazards between agents, which are an important aspect to many systems and especially for airborne systems. In this work, team negotiation is performed using a decentralized gradient-based optimization approach whereas safety distance constraints are specifically designed and handled using Lagrangian multiplier methods. The novelty of our work is the demonstration of a decentralized form of inter-agent collision avoidance in the loop of the agents' real-time group mission optimization process, where the algorithm inherits the properties of performing its original mission while minimizing the probability of inter-agent collisions. Explicit constraint gradient formulation is derived and used to enhance computational advantage and solution accuracy. The effectiveness and robustness of our algorithm has been verified in a simulated environment by coordinating a team of UAVs searching for targets in a large-scale environment.

Industrial Robotics

  • Tool Position Estimation of a Flexible Industrial Robot Using Recursive Bayesian Methods Authors: Axelsson, Patrik; Karlsson, Rickard; Norrlöf, Mikael
    A sensor fusion method for state estimation of a flexible industrial robot is presented. By measuring the acceleration at the end-effector, the accuracy of the arm angular position is improved significantly when these measurements are fused with motor angle observation. The problem is formulated in a Bayesian estimation framework and two solutions are proposed; one using the extended Kalman filter (EKF) and one using the particle filter (PF). The technique is verified on experiments on the ABB IRB4600 robot, where the accelerometer method is showing a significant better dynamic performance, even when model errors are present.
  • A Sensor-Based Approach for Error Compensation of Industrial Robotic Workcells Authors: Tao, Pey Yuen; Yang, Guilin; Tomizuka, Masayoshi
    Industrial robotic manipulators have excellent repeatability while accuracy is significantly poorer. Numerous error sources in the robotic workcell contributes to the accuracy problem. Modeling and identification of all the errors to achieve the required levels of accuracy may be difficult. To resolve the accuracy issues, a sensor based indirect error compensation approach is proposed in this paper where the errors are compensated online via measurements of the work object. The sensor captures a point cloud of the work object and with the CAD model of the work object, the actual relative pose of the sensor frame and work object frame can be established via a point cloud registration. Once this relationship has been established, the robot will be able to move the tool accurately relative to the work object frame near the point of compensation. A data pre-processing technique is proposed to reduce computation time and prevent a local minima solution during point cloud registration. A simulation study is presented to illustrate the effectiveness of the proposed solution.
  • Robot End-Effector Sensing with Position Sensitive Detector and Inertial Sensors Authors: Wang, Cong; Chen, Wenjie; Tomizuka, Masayoshi
    For the motion control of industrial robots, the end-effector performance is of the ultimate interest. However, industrial robots are generally only equipped with motor-side encoders. Accurate estimation of the end-effector position and velocity is thus difficult due to complex joint dynamics. To overcome this problem, this paper presents an optical sensor based on position sensitive detector (PSD), referred as PSD camera, for direct end-effector position sensing. PSD features high precision and fast response while being cost-effective, thus is favorable for real-time feedback applications. In addition, to acquire good velocity estimation, a kinematic Kalman filter (KKF) is applied to fuse the measurement from the PSD camera with that from inertial sensors mounted on the end-effector. The performance of the developed PSD camera and the application of the KKF sensor fusion scheme have been validated through experiments on an industrial robot.
  • Experiments towards Automated Sewing with a Multi-Robot System Authors: Schrimpf, Johannes; Wetterwald, Lars Erik
    In this paper a concept for automated multi-robot-aided sewing is presented. The objective of the work is to demonstrate automatic sewing of 3D-shaped covers for recliners, by assembling two different hide parts with different shapes, using two robots to align the parts during sewing. The system consists of an industrial sewing machine and two real-time controlled Universal Robots 6-axis industrial manipulators. A force feedback system combined with optical edge sensors is evaluated for the control of the sewing process. The force sensors are used to synchronize the velocity and feed rate between the robots and the sewing machine. A test cell was built to determine the feasibility of the force feedback control and velocity synchronization. Experiments are presented which investigate the ability of the robot to feed a hide part into the sewing machine using a force sensor and different strategies for velocity synchronization.
  • Automated Throwing and Capturing of Cylinder-Shaped Objects Authors: Frank, Thorsten; Janoske, Uwe; Mittnacht, Anton; Schroedter, Christian
    A new approach for transportation of objects within production systems by automated throwing and capturing is investigated. This paper presents an implementation, consisting of a throwing robot and a capturing robot. The throwing robot uses a linear and the capturing robot a rotary axis. The throwing robot is capable of throwing cylinder-shaped objects onto a target point with high precision. The capturing robot there smoothly grips the cylinders during flight by means of a rotational movement. In order to synchronize the capturing robot and the cylinder’s pose and velocity, its trajectory has to be modeled as well as the motion sequences of both robots. The throwing and capturing tasks are performed by the robots automatically without the use of any external sensor system.

Human Detection and Tracking

  • Iterative Pedestrian Segmentation and Pose Tracking under a Probabilistic Framework Authors: Li, Yanli
    This paper presents a unified probabilistic framework to tackle two closely related visual tasks: pedestrian segmentation and pose tracking along monocular videos. Although the two tasks are complementary in nature, most previous approaches focus on them individually. Here, we resolve the two problems simultaneously by building and inferring a single body model. More specifically, pedestrian segmentation is performed by optimizing body region with constraint of body pose in a Markov Random Field (MRF), and pose parameters are reasoned about through a Bayesian filtering, which takes body silhouette as an observation cue. Since the two processes are inter-related, we resort to an Expectation-Maximization (EM) algorithm to refine them alternatively. Additionally, a template matching scheme is utilized for initialization. Experimental results on challenging videos verify the framework's robustness to non-rigid human segmentation, cluttered backgrounds and moving cameras.
  • A Connectionist-Based Approach for Human Action Identification Authors: Alazrai, Rami; Lee, C. S. George
    This paper presents a hierarchal, two-layer, connectionist-based human-action recognition system (CHARS) as a first step towards developing socially intelligent robots. The first layer is a K-nearest neighbor (K-NN) classifier that categorizes human actions into two classes based on the existence of locomotion, and the second layer consists of two multi-layer recurrent neural networks that distinguish between subclasses within each class. A pyramid of histograms of oriented gradients (PHOG) descriptor is proposed for extracting local and spatial features. The PHOG descriptor reduces the dimensionality of input space drastically, which results in better convergence for the learning and classification processes. Computer simulations were conducted to illustrate the performance of the proposed CHARS and the role of temporal factor in solving this problem. A widely used KTH human-action database and the human-action dataset from our lab were utilized for performance evaluation. The proposed CHARS was found to perform better than other existing human-action recognition methods and achieved a 95.55% recognition rate.
  • Using Dempster’s Rule of Combination to Robustly Estimate Pointed Targets Authors: Pateraki, Maria; Baltzakis, Haris; Trahanias, Panos
    In this paper we address an important issue in human-robot interaction, that of accurately deriving pointing information from a corresponding gesture. Based on the fact that in most applications it is the pointed object rather than the actual pointing direction which is important, we formulate a novel approach which takes into account prior information about the location of possible pointed targets. To decide about the pointed object, the proposed approach uses the Dempster-Shafer theory of evidence to fuse information from two different input streams: head pose, estimated by visually tracking the off-plane rotations of the face, and hand pointing orientation. Detailed experimental results are presented that validate the effectiveness of the method in realistic application setups.
  • Head-To-Shoulder Signature for Person Recognition Authors: Kirchner, Nathan; Alempijevic, Alen; Virgona, Alexander Joseph
    Ensuring that an interaction is initiated with a particular and unsuspecting member of a group is a complex task. As a first step the robot must effectively, expediently and reliably recognise the humans as they carry on with their typical behaviours (in situ). A method for constructing a scale and viewing angle robust feature vector (from analysing a 3D pointcloud) designed to encapsulate the inter-person variations in the size and shape of the people's head to shoulder region (Head-to-shoulder signature - HSS) is presented. Furthermore, a method for utilising said feature vector as the basis of person recognition via a Support-Vector Machine is detailed. An empirical study was performed in which person recognition was attempted on in situ data collected from 25 participants over 5 days in a office environment. The results report a mean accuracy over the 5 days of 78.15% and a peak accuracy 100% for 9 participants. Further, the results show a considerably better-than-random (1/23 = 4.5%) result for when the participants were: in motion and unaware they were being scanned (52.11%), in motion and face directly away from the sensor (36.04%), and post variations in their general appearance. Finally, the results show the HSS has considerable ability to accommodate for a person's head, shoulder and body rotation relative to the sensor - even in cases where the person is faced directly away from the robot.
  • Bigram-Based Natural Language Model and Statistical Motion Symbol Model for Scalable Language of Humanoid Robots Authors: Takano, Wataru; Nakamura, Yoshihiko
    The language is a symbolic system unique to human being. The acquisition of language, which has its meanings in the real world, is important for robots to understand the environment and communicate with us in our daily life. This paper propose a novel approach to establish a fundamental framework for the robots which can understand language through their whole body motions. The proposed framework is composed of three modules : ``motion symbol", ``motion language model", and ``natural language model". In the motion symbol module, motion data is symbolized by Hidden Markov Models (HMMs). Each HMM represents abstract motion patterns. Then the HMMs are defined as motion symbols. The motion language model is stochastically designed for links between motion symbols and words. This model consists of three layers of motion symbols, latent variables and words. The connections between the motion symbol and the latent state, and between the latent state and the words is denoted by two kinds of probabilities respectively. One connection is represented by the probability that the motion symbol generates the latent state, and the other connection is represented by the probability that the latent state generates the words. Therefore, the motion language model can connect the motion symbols to the words through the latent state. The natural language model stochastically represents sequences of words. In this paper, a bigram, which is a special case of N-gram model, is adopted as the natura
  • Cognitive Active Vision for Human Identification Authors: Utsumi, Yuzuko; Sommerlade, Eric; Bellotto, Nicola; Reid, Ian
    We describe an integrated, real-time multi-camera surveillance system that is able to find and track individuals, acquire and archive facial image sequences, and perform face recognition. The system is based around an inference engine that can extract high-level information from an observed scene, and generate appropriate commands for a set of pan-tiltzoom (PTZ) cameras. The incorporation of a reliable facial recognition into the high-level feedback is a main novelty of our work, showing how high-level understanding of a scene can be used to deploy PTZ sensing resources effectively. The system comprises a distributed camera system using SQL tables as virtual communication channels, Situation Graph Trees for knowledge representation, inference and high-level camera control, and a variety of visual processing algorithms including an on-line acquisition of facial images, and on-line recognition of faces by comparing image sets using subspace distance. We provide an extensive evaluation of this method using our system for both acquisition of training data, and later recognition. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.