How You Are Evaluated
The Roblox problem-solving assessments assessments are designed to measure specific, job-relevant cognitive abilities. Our assessments allow people to demonstrate their cognitive abilities through actively solving complex problems. Testing begins with a small scale pilot where we pressure test our scoring logic and look for any usability issues. We then move to field testing, the goal of field testing is to collect enough data to set scoring parameters and conduct analyses of test bias. This information is needed to evaluate the test performance and fairness. If results from the field test are satisfactory, we then move the test to operational testing where it is used as part of the selection process. Based on the outcomes from this scientifically driven process, your performance on each assessment will be evaluated based on the totality of your ability to develop solutions.
This information is aggregated using a fair, valid, and reliable scoring model to make meaningful estimates about your latent cognitive abilities. Scoring models like these are widely used in modern assessment, and under the model's framework our reliability level is "very good" in relation to industry standards. To ensure the fairness of the assessment, we keep monitoring the score differences by genders and races. Based on our data in field tests, mean score differences between genders and races are within "very small" or "small" effect sizes, which indicates negligible differences and great fairness across gender and race groups; allowing us to make fair and meaningful predictions about your expected performance on the job.