Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent behavior prediction using keypoint data. One of the methods includes obtaining data characterizing a scene in an environment, the data comprising: (i) context data comprising data characterizing historical trajectories of a plurality of agents up to the current time point; and (ii) keypoint data for a target agent; processing the context data using a context data encoder neural network to generate a context embedding for the target agent; processing the keypoint data using a keypoint encoder neural network to generate a keypoint embedding for the target agent; generating a combined embedding for the target agent from the context embedding and the keypoint embedding; and processing the combined embedding using a decoder neural network to generate a behavior prediction output for the target agent that characterizes predicted behavior of the target agent after the current time point.
PEDESTRIAN BEHAVIOR PREDICTION WITH 3D HUMAN KEYPOINTS
2023-05-18
Patent
Electronic Resource
English
IPC: | B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen |
Learning to Detect Good 3D Keypoints
British Library Online Contents | 2018
|FPGA-based detection of SIFT interest keypoints
British Library Online Contents | 2013
|Distinctive Image Features from Scale-Invariant Keypoints
British Library Online Contents | 2004
|SPiKeS: Superpixel-Keypoints structure for robust visual tracking
British Library Online Contents | 2018
|