Your function
Retrieval-augmented generation (RAG) has become more and more popular due to its potential to achieve comparable performance with smaller models compared with large LLM modes. In the RAG architecture, the knowledge is embedded and stored in external vector databases for retrieval of relevant knowledge by the LLMs instead of the model weights during training. To process a user query, the RAG framework first retrieves relevant knowledge from the vector database, and this knowledge is fed to the generator (an LLM) as external knowledge to generate the final answer. Advanced RAG frameworks retrieve the knowledge from the vector database multiple times during the generation. Thus, the vector database plays an important role in the end-to-end performance of RAG such as accuracy and latency. However, RAG also brings new challenges to vector databases as the ever-increasing size of datasets cannot fit into the main memory and thus needs to be offloaded to SSDs.
The aim of this exploratory project is to understand what roles VectorDB plays in the performance of overall RAG pipelines, and how modern fast storage help accelerate the performance of VectorDBs.
Your duties
- Building programs that investigates the differences in latency, throughput, and scalability of different vector database and indexing algorithms.
- Investigating how the performance of vector databases affect the end-to-end performance of RAG.
- Help with building the next generation of vector databases for the RAG workloads.
Your profile
- Bachelor's or Master’s degree in computer science
- Experience with Python
- Experience with Linux
- Experience with database systems, large language models, and RAG is preferred but not mandatory
- Experience with ML frameworks such as PyTorch is preferred but not mandatory
- Fluent in English
As a university, we strive for equal opportunities for all, recognising that diversity takes many forms. We believe that diversity in all its complexity is invaluable for the quality of our teaching, research and service. We are always looking for talent with diverse backgrounds and experiences. This also means that we are committed to creating an inclusive community so that we can use diversity as an asset.
We realise that each individual brings a unique set of skills, expertise and mindset. Therefore we are happy to invite anyone who recognises themselves in the profile to apply, even if you do not meet all the requirements.
What do we offer?
A challenging position in a socially engaged organisation. At VU Amsterdam, you contribute to education, research and service for a better world. And that is valuable. So in return for your efforts, we offer you:
- a salary of minimum € 3.378,00 (scale 10, level 0) and maximum € 3.707,00 (Scale 10, level 2) gross per month.This is based on UFO profile Education and Research Technician - O profile 2. The exact salary depends on your education and experience.
- a position for at least 0.8 FTE. Your employment contract will initially last 1 year.
We also offer you attractive fringe benefits and regulations. Some examples:
- A full-time 38-hour working week comes with a holiday leave entitlement of 232 hours per year. If you choose to work 40 hours, you have 96 extra holiday leave hours on an annual basis. For part-timers, this is calculated pro rata.
- 8% holiday allowance and 8.3% end-of-year bonus
- contribution to commuting expenses
- solid pension scheme (ABP)
- optional model for designing a personalized benefits package
About us
Our group
The scientific programmer will be embedded in the Massivizing Computer Systems (MCS) group, which focuses on research in distributed computing systems and ecosystems, and currently spans over 40 diverse people of which 3 staff. The group aims to contribute to solving high-challenge, high-impact scientific and societal challenges, and has much international and national visibility in research and education in computing systems.
Faculty of Science
Researchers and students at VU Amsterdam’s Faculty of Science tackle fundamental and complex scientific problems to help pave the way for a sustainable and healthy future. From forest fires to big data, from obesity to malnutrition, and from molecules to the moon: we cover the full spectrum of the natural sciences. Our teaching and research have a strong experimentally technical, computational and interdisciplinary nature.
We work on new solutions guided by value-driven, interdisciplinary methodologies. We are committed to research, valorisation and training socially engaged citizens of the world who will make valuable contributions to a sustainable, healthy future.
Are you interested in joining the Faculty of Science? You will join undergraduate students, PhD candidates and researchers at the biggest sciences faculty in the Netherlands. You will combine a professional focus with a broad view of the world. We are proud of our collegial working climate, characterised by committed staff, a pragmatic attitude and engagement in the larger whole. The faculty is home to over 11,000 students enrolled in 40 study programmes. It employs over 1,600 professionals spread across 10 academic departments.
Vrije Universiteit Amsterdam
Vrije Universiteit Amsterdam stands for values-driven education and research. We are open-minded experts with the ability to think freely - a broader mind. Maintaining an entrepreneurial perspective and concentrating on diversity, significance and humanity, we work on sustainable solutions with social impact. By joining forces, across the boundaries of disciplines, we work towards a better world for people and planet. Together we create a safe and respectful working and study climate, and an inspiring environment for education and research. Learn more about our codes of conduct
We are located on one physical campus, in the heart of Amsterdam's Zuidas business district, with excellent location and accessibility. Over 6,150 staff work at the VU and over 31,000 students attend academic education.
Diversity
Diversity is the driving force of VU Amsterdam. VU wants to be accessible and receptive to diversity in disciplines, cultures, ideas, nationalities, beliefs, preferences and worldviews. We believe that trust, respect, interest and differences lead to new insights and innovation, to sharpness and clarity, to excellence and a broader understanding.
We stand for an inclusive community and believe that diversity and internationalisation contribute to the quality of education, research and our services.
Therefore, we are always searching for people whose backgrounds and experience contribute to the diversity of the VU community.
Interested
Are you interested in this position and do you believe that your experience will contribute to the further development of our university? In that case, we encourage you to submit your application.
Submitting a diploma is part of the application process.
Applications received by e-mail will not be considered.
Acquisition in response to this advertisement is not appreciated.
