PhD Position Acoustic Perception for Intelligent Vehicles

Develop AI for intelligent vehicles to make them anticipate unseen nearby traffic through acoustic perception of traffic sounds

JOB DESCRIPTION

The Intelligent Vehicles group at TU Delft is seeking two PhD candidates to investigate the use of acoustic sensors, such as a microphone array, on a (self-driving) vehicle for detection and localization of external sounds from nearby traffic. The project is co-funded and co-supervised by NXP Semiconductors in Leuven.

Currently, Advanced Driver-Assistance Systems (ADAS) and Automated Vehicles (AVs) rely on cameras, radars, and lidar sensors to detect nearby traffic . Nowadays AI and Machine Learning techniques achieve impressive detection performance for these sensors, however they are most useful in unobstructed areas (e.g. highway) within direct line-of-sight, and under specific environmental conditions (day time, no fog, etc.). In contrast, acoustic detection with vehicle-mounted microphones may help anticipate dangerous traffic situations when visibility is limited.

This project aims for completely novel AI approaches to detect nearby traffic using sound. Traffic produces specific acoustic signatures, including explicit sounds (car horns, sirens), but also tire noise of nearby vehicles, or even electric vehicles that are required by law to emit a recognizable sound when driving at low velocity. Using sound as an additional sensor can lead to safer automated driving, or even enable new forms of ADAS. For example, such sounds can indicate approaching vehicles around the corner while driving through narrow streets, or while exiting a blind parking spot. Detecting sirens will help an AV anticipate and localize emergency vehicles.

There are several challenges that will be addressed, divided over two PhD positions:

• Topic A) To identify promising ADAS applications that can benefit from acoustic sensing, design novel use case-specific solutions with Machine Learning and build a working proof-of-concept system to be tested in controlled real-world experiments for at least one application. The novel methods will only need to support the selected ADAS applications and should not extract more information than needed. By aiming for minimal hardware and computational resources, the practical value for ADAS is maximized.
• Topic B) To develop novel acoustic Deep Learning techniques to achieve state-of-the-art performance in multi-sensor environment perception for autonomous driving. The models should provide holistic representations of all surrounding traffic by fusing acoustic with other sensor modalities. Such holistic traffic representations support many driving tasks at once, as required for full self-driving. The addition of acoustics should improve the robustness of the existing sensor suite.

The IV group’s Prius demonstrator vehicle with cameras, lidars, radars, and a roof-mounted microphone array will be used to investigate acoustic detection and localization salient traffic sounds. The project aims for viable real-time solutions and an online demonstration of acoustic ADAS

REQUIREMENTS

• Completed (or about to complete) a MSc degree related to any of: artificial intelligence, machine learning, intelligent vehicles / robotics, acoustics and signal processing, computer vision.
• Good theoretic understanding of the fundamentals of Machine Learning.
• Ability to act independently as well as to collaborate effectively with members of a larger interdisciplinary team, take initiative, be result oriented, organized and creative.
• Excellent programming skills (Python, Matlab or C/C++).
• Good command of verbal and written English.

Desirable for topic A:

• Hands-on experience with real-world data collection in robotics, intelligent vehicles, or a related area.
• Experience in applying Machine Learning / Deep Learning to real-world data.
• Knowledge of acoustic signal processing (filtering, beamforming, direction of arrival, SRP-PHAT) is a plus.
• Knowledge of the Robot Operating System (ROS) is a plus.

Desirable for topic B:

• Demonstratable experience in applying Deep Learning, using PyTorch, TensorFlow, JAX.
• Experience with robotic/vehicle perception, preferably acoustics, but experience with computer vision, lidar, radar perception is also a plus.

Doing a PhD at TU Delft requires English proficiency at a certain level to ensure that the candidate is able to communicate and interact well, participate in English-taught Doctoral Education courses, and write scientific articles and a final thesis. For more details please check the Graduate Schools Admission Requirements.

CONDITIONS OF EMPLOYMENT

Fixed-term contract: 4 years.

Doctoral candidates will be offered a 4-year period of employment in principle, but in the form of 2 employment contracts. An initial 1,5 year contract with an official go/no go progress assessment within 15 months. Followed by an additional contract for the remaining 2,5 years assuming everything goes well and performance requirements are met.

Salary and benefits are in accordance with the Collective Labour Agreement for Dutch Universities, increasing from € 2770 per month in the first year to € 3539 in the fourth year. As a PhD candidate you will be enrolled in the TU Delft Graduate School. The TU Delft Graduate School provides an inspiring research environment with an excellent team of supervisors, academic staff and a mentor. The Doctoral Education Programme is aimed at developing your transferable, discipline-related and research skills.

The TU Delft offers a customisable compensation package, discounts on health insurance, and a monthly work costs contribution. Flexible work schedules can be arranged.

For international applicants, TU Delft has the Coming to Delft Service. This service provides information for new international employees to help you prepare the relocation and to settle in the Netherlands. The Coming to Delft Service offers a Dual Career Programme for partners and they organise events to expand your (social) network.

EMPLOYER

Delft University of Technology

Delft University of Technology is built on strong foundations. As creators of the world-famous Dutch waterworks and pioneers in biotech, TU Delft is a top international university combining science, engineering and design. It delivers world class results in education, research and innovation to address challenges in the areas of energy, climate, mobility, health and digital society. For generations, our engineers have proven to be entrepreneurial problem-solvers, both in business and in a social context.

At TU Delft we embrace diversity as one of our core values and we actively engage to be a university where you feel at home and can flourish. We value different perspectives and qualities. We believe this makes our work more innovative, the TU Delft community more vibrant and the world more just. Together, we imagine, invent and create solutions using technology to have a positive impact on a global scale. That is why we invite you to apply. Your application will receive fair consideration.

Challenge. Change. Impact!

DEPARTMENT

Faculty Mechanical Engineering

From chip to ship. From machine to human being. From idea to solution. Driven by a deep-rooted desire to understand our environment and discover its underlying mechanisms, research and education at the ME faculty focusses on fundamental understanding, design, production including application and product improvement, materials, processes and (mechanical) systems.

ME is a dynamic and innovative faculty with high-tech lab facilities and international reach. It’s a large faculty but also versatile, so we can often make unique connections by combining different disciplines. This is reflected in ME’s outstanding, state-of-the-art education, which trains students to become responsible and socially engaged engineers and scientists. We translate our knowledge and insights into solutions to societal issues, contributing to a sustainable society and to the development of prosperity and well-being. That is what unites us in pioneering research, inspiring education and (inter)national cooperation.

Click here to go to the website of the Faculty of Mechanical Engineering. Do you want to experience working at our faculty? These videos will introduce you to some of our researchers and their work.

ADDITIONAL INFORMATION

For more information about this vacancy, please contact Julian Kooij via J.F.P.Kooij@tudelft.nl.