Overview
Roblox prides itself on taking the long view to develop a culture of innovation throughout our workforce. Hiring selection is no different. Roblox’s game-based cognitive assessments use state-of-the-art scoring techniques alongside engaging and interactive elements from Game Design, 3D Art, Engineering, and UX/UI Design to bring to life a one-of-a-kind, user-centric testing experience that is immersive and engaging. This, combined with a CodeSignal coding assessment, help us understand the candidate's abilities and provide additional data points on their application.
The Roblox cognitive selection assessments are designed to evaluate candidates' problem solving abilities within game-based environments. This type of thinking is critical for virtually all employees’ daily work and our hiring assessments are developed and tested specifically for Roblox and the needs of our workforce. The assessment development and testing process is guided by rigorous scientific frameworks and best practices from the fields of Learning Science, Psychometrics, and Data Science.
Assessment development at Roblox is driven by rigorous scientific frameworks and a continuous validation process. We develop our assessments specifically for Roblox and the needs of our workforce. We begin development with a series of interviews aimed at better understanding the knowledge, skills, and abilities (KSAs) needed for success at Roblox across our various job families. From there, we develop initial concepts and designs aimed to capture evidence of these specific KSAs. Through a series of iterations and evaluations we select the best two to three concepts to paper prototype, eventually selecting one to turn into a full digital assessment.
Once we have a concept that we will turn into a digital assessment we begin a series of in-depth user tests. We start our testing approach by conducting think-alouds (TAs). These TAs help us elicit information about usability as well as construct alignment. Once we have a stable digital assessment build we begin our testing protocol.
Why this Assessment and these Skills
When evaluating candidates for roles at Roblox, we are interested not only in strong analytical skills and coding ability, but also in the application of those skills and abilities in higher-order thinking. Research at Roblox has demonstrated that complex skills such as problem solving and systems thinking are necessary for success in our roles, and high levels of ability in these areas indicates potential to make a long-term positive impact at the company. We want to make sure that each candidate is given the opportunity to demonstrate these skills and abilities that are important to their prospective role. The use of an assessment provides an equitable opportunity for all candidates, regardless of background to demonstrate job relevant skills. By evaluating a candidate's skills and abilities using a fair, reliable, and valid assessment, we can identify those who may perform at the level needed to be successful in their role.
What to Expect
Throughout the Roblox cognitive selection process you will be presented with a series of different surveys and assessments to complete. Our assessments will ask you to demonstrate a range of varying cognitive skills as you work to solve tasks effectively and efficiently.
Our hiring process begins with a link to your personalized assessment, which will direct you to a list of assigned tasks to complete. Before you reach this list, you will be asked to agree to several terms and conditions; please review these carefully as they are legally binding. You may also be asked to…
- Share your Roblox username, in the event you have been assigned one or more Roblox-based tasks. If you don’t have a username, you can create one and download the Roblox application on Roblox.com.
- Complete a quick systems check, which will verify if your computer meets the technical requirements for your assessment.
After completing these steps, you will be directed to your assessment dashboard where you will see your assigned tasks, along with your deadline to complete those tasks.
You will begin each cognitive skills task by completing its respective tutorial. Roblox tutorials are designed to provide you with all of the necessary information you will need to understand the structure, controls, and goals to be successful within an allotted amount of time. Importantly, once you begin a cognitive skills task, you will have a fixed amount of time to complete it. This time limit continues even if you close out of the ongoing task. So be aware that once you begin a task, you will need to be able to complete it within the specified amount of time. Be mindful of this requirement, and allow yourself enough time to complete a task before you begin.
Some, but not all, Roblox cognitive skills tasks are broken up into a series of sections to complete. These separated sections typically resemble each other, but each requires you to use various problem solving skills in order to succeed. Any changes to the goal for a task will be explicitly stated, informing you about what is expected of you during each section of a task. Across all of our Roblox tasks, the format of each task may look quite different from one to the next as each task is designed to test different aspects of your cognitive abilities.
How You Are Evaluated
The Roblox cognitive assessments are designed to measure specific, job-relevant cognitive abilities. Our assessments allow people to demonstrate their cognitive abilities through actively solving complex problems. Testing begins with a small scale Pilot where we pressure test our scoring logic and look for any usability issues. We then move to field testing, the goal of field testing is to collect enough data to set scoring parameters and conduct analyses of test bias. This information is needed to evaluate the test performance and fairness. If results from the field test are satisfactory, we then move the test to operational testing where it is used as part of the selection process. Based on the outcomes from this scientifically driven process, your performance on each assessment will be evaluated based on the totality of your ability to develop solutions.
This information is aggregated using a fair, valid, and reliable scoring model to make meaningful estimates about your latent cognitive abilities. Scoring models like these are widely used in modern assessment, and under the model's framework our reliability level is "very good" in relation to industry standards. To ensure the fairness of the assessment, we keep monitoring the score differences by genders and races. Based on our data in field tests, mean score differences between genders and races are within "very small" or "small" effect sizes, which indicates negligible differences and great fairness across gender and race groups. Allowing us to make fair and meaningful predictions about your expected performance on the job.